Data Engineering Lead Analyst - HIH - Evernorth

Key Responsibilities:

• Design, implement, and optimize data pipelines using Apache Spark and Databricks to ingest, process, and transform large-scale structured and unstructured datasets.
• Develop, schedule, and monitor ETL workflows and DAGs using orchestration tools such as Airflow, Jenkins, or similar.
• Build and maintain data models, tables, and schemas optimized for analytics and reporting in cloud-based data warehouses.
• Write clean, reusable, and modular Python code following best practices and standards.
• Use AWS Glue for metadata management, data cataloging, and ETL transformation jobs.
• Collaborate with analytics and product teams to translate data needs into engineering solutions.
• Ensure data quality, reliability, and observability through unit testing, logging, alerting, and documentation.
• Support version control and CI/CD practices for data pipelines.
• Participate in code reviews and contribute to team knowledge sharing.