Lead Data Engineer

Job Role : Senior / Lead Data Engineer

• Experience Level: 4 – 10 Years

• Employment Type: Full-time

• Location: Hybrid / Remote / Onsite

Role Purpose

• As a Senior Data Engineer, you will own the end-to-end data pipelines that feed our analytics and production machine learning models. You are responsible for ensuring that massive volumes of raw data arrive reliably, scale gracefully, and can be trusted across the entire enterprise. • Key Responsibilities

• Architect, implement, and maintain scalable batch and real-time streaming data pipelines using Apache Spark, Kafka, and Python/Scala.

• Design and manage cloud data warehouse and lakehouse architectures (Snowflake, Databricks, BigQuery).

• Implement robust pipeline orchestration using Apache Airflow or Prefect.

• Enforce data governance, quality frameworks, schema evolution, and automated unit/integration testing on data assets.

• Partner with Data Scientists and ML Engineers to supply reliable, low-latency features and ingestion workflows.

Qualifications & Requirements

• Relevant Experience: 4–10 years of dedicated software engineering and data pipeline development experience.

• Core Technical Stack: Advanced Python and/or Scala, expert SQL, Spark (PySpark), Kafka/Kinesis, Airflow, and Snowflake/Databricks.

• Proven Track Record: Tangible history of processing multi-terabyte/petabyte workloads and handling millions of events/day in production.

• Engineering Standards: Deep understanding of containerization (Docker), version control (Git), and CI/CD for data pipelines.