We are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms.

Key Responsibilities

Design, develop, and optimize scalable data pipelines using
Databricks (PySpark/Spark SQL)
.
Implement and tune data workloads using
Azure Data Factory
,
Azure Synapse Analytics
, and
Microsoft Fabric
.
Apply best practices in
Delta Lake
, including performance tuning, partitioning, indexing, and caching.
Troubleshoot complex ingestion, transformation, orchestration, and performance issues.
Implement data governance, lineage, and security using
Unity Catalog
and Azure capabilities.
Automate workflows using CI/CD, Databricks Jobs, and REST APIs.
Collaborate with data scientists, analysts, and business teams to deliver clean and reliable data.
Monitor platform health, optimize cluster configurations, and ensure cost-efficient scaling.

Required Skills & Qualifications

3–5 years
of data engineering experience.
Strong hands-on development skills on
Databricks
in production environments.
Expertise in
PySpark / Spark SQL
and distributed data processing.
Professional experience with:

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

Strong knowledge of:
Delta Lake ACID transactions & schema evolution
ETL/ELT architecture and data warehousing concepts
Performance tuning of Spark jobs and pipeline optimization
Cloud platform experience —
Azure preferred
.
Proficiency with Git-based version control and DevOps practices.

Preferred Certifications

Databricks Certified Data Engineer Associate or Professional — Mandatory
Azure Data Engineering certifications (DP-203 or equivalent) — Advantageous

Soft Skills

Strong analytical and root-cause troubleshooting abilities.
Excellent communication, documentation, and stakeholder coordination.
Ability to work independently and within cross-functional agile teams.

Why Join Us?

Work with leading-edge Azure and Databricks technologies.
Drive impactful data initiatives with high autonomy.
Growth-oriented environment with continuous learning opportunities.

More Jobs at Senpiper PTY LTD

Senior Data & Analytics Lead

gurugram, haryana, india

10.0 - 10.0 yrs

Salary: Not disclosed

Data Engineer

gurugram, haryana, india

5.0 - 5.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.