Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in hyderabad
>
Toppan Merrill
>
Senior Data Engineer

Senior Data Engineer

Toppan Merrill

3 - 6 years

6 - 10 Lacs

hyderabad gurugram chennai

Posted:1 month ago| Platform:

Apply

Skills Required

access control performance tuning metadata spark debugging data governance data quality monitoring sql python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Develop & Optimize Data Pipelines

Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark) .
Optimize performance and cost-efficiency of Spark jobs.
Ensure data quality through validation, monitoring, and alerting mechanisms.
Understand cluster types, configuration, and use-case for serverless

Implement Unity Catalog for Data Governance

Design and enforce access control policies using Unity Catalog.
Manage data lineage, auditing, and metadata governance .
Enable secure data sharing across teams and external stakeholders.

Integrate with Cloud Data Platforms

Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub to integrate Databricks with cloud-based data lakes, data warehouses, and event streams .
Implement Delta Lake for scalable, ACID-compliant storage.

Automate & Orchestrate Workflows

Develop CI/CD pipelines for data workflows using Azure Databricks Workflows or Azure Data Factory .
Monitor and troubleshoot failures in job execution and cluster performance .

Collaborate with Stakeholders

Work with Data Analysts, Scientists, and Business Teams to understand requirements.
Translate business needs into scalable data engineering solutions .

API expertise

Ability to pull data from a wide variety of APIs using different strategies and methods

Required Skills & Experience:

Azure Databricks & Apache Spark (PySpark) Strong experience in building distributed data pipelines .
Python Proficiency in writing optimized and maintainable Python code for data engineering.
Unity Catalog Hands-on experience implementing data governance, access controls, and lineage tracking .
SQL Strong knowledge of SQL for data transformations and optimizations.
Delta Lake Understanding of time travel, schema evolution, and performance tuning .
Workflow Orchestration Experience with Azure Databricks Jobs or Azure Data Factory .
CI/CD & Infrastructure as Code (IaC) Familiarity with Databricks CLI, Databricks DABs, and DevOps principles .
Security & Compliance Knowledge of IAM, role-based access control (RBAC), and encryption .

Preferred Qualifications:

Experience with MLflow for model tracking & deployment in Databricks.
Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).
Hands-on experience with dbt (Data Build Tool) for modular ETL development.
Certification in Databricks, Azure is a plus.
Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server
Experience with Azure Synapse Link for Dynamics, dataverse
Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc

Soft Skills:

Strong problem-solving and debugging skills.
Ability to work independently and in teams .
Excellent communication and documentation skills.

More Jobs at Toppan Merrill

Oracle EBS Technical Developer

Chennai

5 - 7 yrs

INR 15 - 22 Lacs

Adobe InDesign template designer

Chennai, Tamil Nadu

Experience: Not specified

Salary: Not disclosed

Sr. Technical Analyst / Architect

Greater Chennai Area

Experience: Not specified

Salary: Not disclosed

Software Developer

Greater Chennai Area

Experience: Not specified

Salary: Not disclosed

Quality Analyst - Manual

Greater Chennai Area

4.0 - 8.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.