Lead Data Engineer

Key Responsibilities

• Lead and mentor data engineering, data science, and analytics team members.
• Architect and implement scalable end-to-end data pipelines and distributed data processing.
• Design and optimize ETL/ELT workflows using Airflow, Spark/PySpark, and related tools.
• Develop and manage REST APIs and microservices and integrate external systems and tools.
• Build and maintain cloud infrastructure with strong focus on AWS services.
• Manage containerization and orchestration using Docker and advanced Kubernetes.
• Implement and manage enterprise data warehousing solutions, preferably Snowflake.
• Support research and implementation of GenAI-based solutions.
• Define and enforce data standards, governance, and data quality frameworks.
• Collaborate with business and product teams to deliver data-driven insights.

Education and Experience

• Bachelor’s degree in Computer Science, Engineering, or related field.
• Minimum 7+ years of relevant experience in data engineering, data science, or software development.
• 5+ years hands-on experience with Python and SQL.
• 5+ years experience in AWS cloud services.
• Strong Docker expertise (5+ years) and advanced Kubernetes experience.
• Airflow experience (5+ years) for workflow orchestration.
• Hands-on experience with Spark / PySpark.
• Data warehousing experience (Snowflake preferred).
• Basic understanding of ELK stack.
• Experience with system integrations (Slack, Glean AI, OpenMetadata preferred).
• Intermediate experience with GenAI / LLM-based implementations.

Analytical and Personal Skills

• Stakeholder management and strong verbal & written communication skills are a must.
• Strong leadership, analytical thinking, and problem-solving capabilities.
• Self-driven, motivated and detail oriented.
• Patient, persistent and focused