Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in chennai
>
Toppan Merrill
>
Senior Azure Data Engineer

Senior Azure Data Engineer

Toppan Merrill

7 - 11 years

19 - 34 Lacs

chennai

Posted:2 hours ago| Platform:

Apply

Skills Required

azure databricks azure data factory azure synapse pyspark delta lake unity catalog python sql

Work Mode

Hybrid

Job Type

Full Time

Job Description

Seeking a Data Engineer with expertise in Azure Databricks, Apache Spark (Python), and Unity Catalog to design, develop, and optimize data pipelines, ensuring secure and scalable data governance.

Job Description

Responsibilities:

Develop & Optimize Data Pipelines

Build, test, and maintain
ETL/ELT
data pipelines using Azure
Databricks & Apache Spark (PySpark)
.
Optimize
performance and cost-efficiency
of Spark jobs.
Ensure data quality through validation, monitoring, and alerting mechanisms.
Understand cluster types, configuration, and use-case for serverless

Implement Unity Catalog for Data Governance

Design and enforce
access control policies
using Unity Catalog.
Manage
data lineage, auditing, and metadata governance
.
Enable secure data sharing across teams and external stakeholders.

Integrate with Cloud Data Platforms

Work with
Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub
to integrate Databricks with cloud-based
data lakes, data warehouses, and event streams
.
Implement
Delta Lake
for scalable, ACID-compliant storage.

Automate & Orchestrate Workflows

Develop
CI/CD
pipelines for data workflows using
Azure

Databricks Workflows or Azure Data Factory
.
Monitor and troubleshoot failures in
job execution and cluster performance
.

Collaborate with Stakeholders

Work with
Data Analysts, Scientists, and Business Teams
to understand requirements.
Translate business needs into
scalable data engineering solutions
.

API expertise

Ability to pull data from a wide variety of APIs using different strategies and methods

Required Skills & Experience:

Azure Databricks & Apache Spark (PySpark)
Strong experience in building
distributed data pipelines
.
Python
Proficiency in writing optimized and maintainable Python code for data engineering.
Unity Catalog
– Hands-on experience implementing
data governance, access controls, and lineage tracking
.
SQL
– Strong knowledge of SQL for data transformations and optimizations.
Delta Lake
– Understanding of
time travel, schema evolution, and performance tuning
.
Workflow Orchestration
– Experience with
Azure

Databricks Jobs or Azure Data Factory
.
CI/CD & Infrastructure as Code (IaC)
– Familiarity with
Databricks CLI, Databricks DABs, and DevOps principles
.
Security & Compliance
– Knowledge of
IAM, role-based access control (RBAC), and encryption
.

Preferred Qualifications:

Experience with
MLflow
for model tracking & deployment in Databricks.
Familiarity with
streaming technologies
(Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).
Hands-on experience with
dbt(Data Build Tool)
for modular ETL development.
Certification in
Databricks, Azure is
a plus.
Experience with Azure Databricks Lakehouse connectors for Salesforce and SQL Server
Experience with Azure Synapse Link for Dynamics, Dataverse
Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc.

Soft Skills:

Strong
problem-solving
and debugging skills.
Ability to
work independently and in teams
.
Excellent
communication and documentation
skills.

More Jobs at Toppan Merrill

Oracle EBS Technical Developer

Chennai

5 - 7 yrs

INR 15 - 22 Lacs

Adobe InDesign template designer

Chennai, Tamil Nadu

Experience: Not specified

Salary: Not disclosed

Sr. Technical Analyst / Architect

Greater Chennai Area

Experience: Not specified

Salary: Not disclosed

Software Developer

Greater Chennai Area

Experience: Not specified

Salary: Not disclosed

Quality Analyst - Manual

Greater Chennai Area

4.0 - 8.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.