ETL Data Engineer – AI/ML & LLM

ETL Data Engineer – AI/ML & LLM

Experience: 7–8 Years

Work Location: Hybrid – PAN India

Job Type: Full-Time / Contract (As Applicable)

Job Summary

We are looking for an experienced ETL Data Engineer with strong expertise in Azure Databricks, Python, Apache Spark, and AI/ML/LLM technologies.
The ideal candidate should have hands-on experience designing, developing, deploying, and optimizing scalable ETL/ELT data pipelines to support enterprise analytics, business intelligence, and machine learning solutions.

Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, Lakebase, Python, and Apache Spark.

- Build and optimize data pipelines for analytics, BI reporting, machine learning, and predictive modelling.

- Integrate data from multiple sources, including databases, APIs, FTP, and cloud storage.

- Develop complex SQL queries and perform query and performance optimization.

- Ensure data quality, integrity, reliability, and pipeline efficiency.

- Work on data architecture, data modelling, and cloud-based data engineering solutions.

- Implement CI/CD and DevOps automation using Terraform and GitHub Actions.

- Collaborate with AI/ML teams to integrate LLM-based solutions.

- Provide technical guidance and mentor junior data engineers.

Required Skills & Experience

- 7+ years of experience in Data Engineering and ETL/ELT development.

- 5+ years of hands-on experience with Python.

- 2+ years of experience with Apache Spark, PySpark, and Spark SQL.

- 2+ years of experience with Azure Databricks, including Unity Catalog.

- Practical experience with AI/ML and LLM technologies.

- Strong knowledge of Advanced SQL, data modelling, and data warehousing.

Additional Technical Skills

- Azure Functions and Azure Service Bus.

- Snowflake and Lakebase.

- Delta Lake, Data Lake, and Data Warehousing.

- Terraform and DevOps automation.

- CI/CD tools: GitHub Actions, Git, Artifactory, and Sonar.

- Experience with cloud-based data engineering architectures.

Pay: ₹90,000.00 - ₹1,600,000.00 per month

Experience:

• Data Engineering: 7 years (Required)

Work Location: Hybrid remote in Noida, Uttar Pradesh