Lead Data Engineer - Azure Databricks & DevOps

Blend360 India

7 - 12 years

12 - 17 Lacs

hyderabad

Posted:2 weeks ago| Platform:

Apply

Skills Required

performance tuning automation version control git data security analytical debugging analytics sql python

Work Mode

Work from Office

Job Type

Full Time

Job Description

We are looking for a highly skilled

Senior Data Engineer

with a strong background in

Azure Databricks, Apache Spark, Python, SQL,

and

DevOps

practices. The ideal candidate will design and implement robust, scalable, and high-performing data solutions while ensuring smooth deployment and automation through CI/CD frameworks.

Responsibilities

Design, develop, and optimize
data pipelines
and
ETL workflows
using
Azure Databricks
and
Apache Spark (PySpark)
.
Build scalable
data ingestion
and
transformation
frameworks integrating multiple structured and unstructured data sources.
Implement and manage
DevOps practices
for data workflows, including CI/CD, version control, and automation.
Collaborate with cross-functional teams to deploy and monitor data applications in Azure environments.
Write efficient and reusable
Python
and
SQL
scripts for data cleaning, transformation, and analytics.
Implement
data lakehouse
architectures using
Azure Data Lake
and
Delta Lake
.
Ensure performance tuning, cost optimization, and adherence to data security best practices.
Troubleshoot, monitor, and resolve production issues related to data pipelines and infrastructure

6+ years
of experience in
Data Engineering
or related domains.
Strong hands-on experience in
Azure Databricks
,
Apache Spark
, and
PySpark
.
Advanced proficiency in
Python
and
SQL
for data engineering tasks.
Experience with
DevOps tools and concepts
, including:
- CI/CD pipelines (Azure DevOps, GitHub Actions, or Jenkins)
- Version control (Git)
- Infrastructure as Code (ARM templates, Terraform preferred)
Solid understanding of
Azure Data Factory
,
Azure Data Lake
, and
Azure Synapse
.
Proven experience in
data pipeline orchestration
,
monitoring
, and
automation
.
Strong analytical, problem-solving, and debugging skills.
Ability to work in a fast-paced, collaborative environment with cross-functional teams.

More Jobs at Blend360 India

Senior Data Scientist - Forecasting

Hyderabad

5 - 7 yrs

INR 20 - 27 Lacs

Lead Data Engineer

Hyderabad

5 - 10 yrs

INR 15 - 30 Lacs

Lead Azure DevOps Engineer

Hyderabad

7 - 12 yrs

INR 15 - 25 Lacs

Senior AWS Data Engineer

Hyderabad

5 - 10 yrs

INR 15 - 30 Lacs

Lead Azure Data Engineer

Hyderabad

6 - 11 yrs

INR 15 - 25 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.