Posted:2 days ago|
Platform:
On-site
Full Time
· Define the end-to-end data engineering strategy and technical vision aligned with business goals.
· Architect scalable, high-performance, and cost-efficient data platforms Preferably on (AWS, Azure) or GCP.
· Design and optimize data Lakehouse medallion architectures integrating batch and streaming pipelines (Spark, Kafka, Delta Lake, Iceberg, Hudi, etc.).
· Build reusable frameworks for data ingestion, transformation, and orchestration across heterogeneous systems
· Lead the development and optimization of ETL/ELT pipelines using PySpark, Scala, SQL, and Airflow or equivalent orchestration tools.
· Oversee the implementation of real-time streaming solutions leveraging Kafka, Kinesis, or Pub/Sub. Guide the team in integrating structured, semi-structured, and unstructured data sources.
· Drive adoption of DataOps and DevOps best practices — CI/CD for data pipelines, automated testing, and monitoring
· Design and manage cloud-native data solutions (AWS Glue, EMR, Redshift, Snowflake, BigQuery, Databricks, etc.)
· Lead initiatives for data platform modernization and cloud migration strategies.
· Define and enforce standards for data quality, lineage, governance, and metadata management (e.g., Great Expectations, Apache Atlas, or Collibra).
· Implement robust data security, compliance, and privacy frameworks aligned with industry standards (GDPR, HIPAA, etc.).
· Establish observability frameworks for data pipelines — logging, monitoring, and anomaly detection
· Lead and mentor a team of senior and mid-level data engineers, fostering a culture of excellence, ownership, and innovation.
· Collaborate cross-functionally with AI/ML, Analytics, and Product Engineering teams to enable data-driven decision-making.
· Evaluate emerging technologies (e.g., VectorDBs, GraphDBs, RAG frameworks) and drive their adoption for advanced data-driven use cases.
· Represent the data engineering practice in architecture reviews and executive technology forums.
· 10+ years in data engineering and architecture roles, with 3+ years in technical leadership or data platform lead roles.
Deep proficiency in Spark, PySpark, Scala, SQL, and distributed data processing.
· Strong experience with big data ecosystems (Hadoop, Hive, Presto, Trino, Delta Lake, etc.).
· Proven hands-on work in cloud data platforms – AWS (Glue, EMR, Redshift), GCP (Dataflow, BigQuery), or Azure (Synapse, Data Factory).
· Experience with workflow orchestration (Airflow, Dagster, Prefect) and containerization (Docker, Kubernetes).
· Expertise in modern data storage systems – Snowflake, Databricks, Iceberg, Hudi, or Delta Lake.
· Exposure to data observability, data mesh, and feature store frameworks.
· Strong people management, mentorship, and cross-functional collaboration skills.
· Demonstrated success in building or scaling a data engineering function or CoE (Center of Excellence).
· AWS Certified Data Analytics – Specialty / GCP Professional Data Engineer / Databricks Certified Data Engineer / Snowflake Architect.
TalentLynk
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
navi mumbai
6.0 - 13.0 Lacs P.A.
chennai
8.0 - 11.0 Lacs P.A.
hyderabad, chennai
11.0 - 21.0 Lacs P.A.
bengaluru
0.5 - 3.0 Lacs P.A.
noida, navi mumbai, bengaluru
18.0 - 32.5 Lacs P.A.
vadodara
3.0 - 6.0 Lacs P.A.
chennai
8.0 - 11.0 Lacs P.A.
hyderabad
5.0 - 6.0 Lacs P.A.
bengaluru
0.5 - 0.6 Lacs P.A.
bengaluru
5.0 - 9.0 Lacs P.A.
