ETL Data Engineer

As an ETL Data Engineer, your role will involve re-engineering existing data pipelines to extract data from a new source (PostgreSQL / CURA system) instead of Microsoft SQL Server while maintaining data quality, system performance, and end-user experience during migration. Key Responsibilities: - Analyze current ETL pipelines and their dependencies on Microsoft SQL Server. - Design and implement modifications to redirect ETL extractions to PostgreSQL (CURA) while keeping current transformations and load logic for Elasticsearch and MongoDB. - Ensure end-to-end data integrity, quality, and freshness post source switch. - Write efficient SQL queries for data extraction from the new source. - Perform performance testing to validate no degradation in pipeline throughput or latency. - Collaborate with DevOps and platform teams to containerize, orchestrate, and deploy updated ETLs using Docker and Kubernetes. - Monitor post-deployment performance and proactively address any production issues. - Document design, code, data mappings, and operational runbooks. Skills and Qualifications Required: - Strong experience in building and maintaining large-scale distributed data systems. - Expertise in Python, specifically data analysis/manipulation libraries like pandas, NumPy, and Polars. - Advanced SQL development skills with a focus on performance optimization. - Proficiency in Docker and Kubernetes. - Familiarity with Elasticsearch and MongoDB as data stores. - Experience working in production environments with mission-critical systems.,