Lead Data Engineer

We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex data initiatives, guiding engineering teams, and collaborating with cross-functional stakeholders to deliver robust, scalable, and secure data solutions.

• Design and build scalable ETL/ELT pipelines using Python / PySpark.
  

• Architect and implement batch and real-time data pipelines using Kafka, Kafka Connect, and Debezium.
  

• Develop and optimize data ingestion workflows across multiple databases (MySQL, Postgres, MongoDB).
  

• Build and manage Apache Airflow DAGs for end-to-end pipeline orchestration.
  

• Lead data platform development on GCP (BigQuery, Data Proc, Data Stream ).
  

• Implement data ingestion, orchestration, and transformation at scale using Pub/Sub, Cloud Functions/Cloud Run, Dataflow, and GKE.
  

• Own data warehouse design and optimization across Snowflake and BigQuery.
  

• Strong proficiency in Python, SQL, PySpark.
  

• Hands-on expertise with Kafka, Kafka Connect, Debezium, Airflow, Databricks.
  

• Deep experience with BigQuery, Snowflake, MySQL, Postgres, MongoDB.
  

• Solid understanding of vector data stores and search indexing.
  

• Knowledge of GCP services like Big Query, Cloud Functions, Cloud Run, Data Flow, Data Proc, Data Stream, etc..
  

• Certifications:
  

• GCP Professional Data Engineer
  

• Elastic Certified Engineer
  

• AI
  

• Gemini Enterprise
  

• Vertex AI Agent Builder
  

• ADK
  

• Communication: Exceptional verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.
  

• Mentorship & Coaching: Proven experience in mentoring junior and mid-level engineers, fostering a culture of continuous learning and growth.
  

• Problem-Solving: Strong analytical and debugging skills, with a proactive approach to identifying and resolving technical roadblocks.
  

• Ownership & Accountability: Demonstrates a high level of responsibility for project outcomes, system reliability, and code quality.
  

• Agile Proficiency: Deep understanding and practical experience with Agile methodologies (Scrum/Kanban).
  

• Stakeholder Management: Ability to effectively manage expectations and build consensus across different teams.
  

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent practical experience).