Home
Jobs
Companies
Resume
Celebal Technologies logo

Azure Data Engineer

7 - 12 years

20 - 35 Lacs

Mumbai

Posted:2 weeks ago| Platform: Naukri logo

Apply

Skills Required

Streaming Pyspark Kafka Data Bricks Autoloader Airflow Screaming Frog Integration Data Engineering Azure Databricks Medallion Architecture Apache SQL Azure Data Factory Azure Cloud SCALA flatmap Delta Lake Microsoft Azure Cluster Optimization Spark Map Python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Summary: We are looking for a highly skilled Azure Data Engineer with a strong background in real-time and batch data ingestion and big data processing, particularly using Kafka and Databricks . The ideal candidate will have a deep understanding of streaming architectures , Medallion data models , and performance optimization techniques in cloud environments. This role requires hands-on technical expertise , including live coding during the interview process. Key Responsibilities Design and implement streaming data pipelines integrating Kafka with Databricks using Structured Streaming . Architect and maintain Medallion Architecture with well-defined Bronze, Silver, and Gold layers . Implement efficient ingestion using Databricks Autoloader for high-throughput data loads. Work with large volumes of structured and unstructured data , ensuring high availability and performance. Apply performance tuning techniques such as partitioning, caching , and cluster resource optimization . Collaborate with cross-functional teams (data scientists, analysts, business users) to build robust data solutions. Establish best practices for code versioning , deployment automation , and data governance . Required Technical Skills: Strong expertise in Azure Databricks and Spark Structured Streaming Processing modes (append, update, complete) Output modes (append, complete, update) Checkpointing and state management Experience with Kafka integration for real-time data pipelines Deep understanding of Medallion Architecture Proficiency with Databricks Autoloader and schema evolution Deep understanding of Unity Catalog and Foreign catalog Strong knowledge of Spark SQL, Delta Lake, and DataFrames Expertise in performance tuning (query optimization, cluster configuration, caching strategies) Must have Data management strategies Excellent with Governance and Access management Strong with Data modelling, Data warehousing concepts, Databricks as a platform Solid understanding of Window functions Proven experience in: Merge/Upsert logic Implementing SCD Type 1 and Type 2 Handling CDC (Change Data Capture) scenarios Retail/Telcom/Energy any one industry expertise Real time use case execution Data modelling Location: Mumbai

More Jobs at Celebal Technologies

Nodejs Full-Stack Developer

Jaipur

3 - 6 yrs
|

INR 0 - 3 Lacs

Hr Analytics Specialist

Jaipur

4 - 8 yrs
|

INR 5 - 8 Lacs

Urgently Looking For Data Scientist|| Immediate Joiner|| Celebal Tech

Delhi NCR, Jaipur

3 - 7 yrs
|

INR 15 - 25 Lacs

Qa Automation Testing Engineer

Delhi NCR, Jaipur

2 - 5 yrs
|

INR 5 - 7 Lacs

Technical Recruiter

Pune

3 - 5 yrs
|

INR 4 - 8 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Streaming Interview Now

My Connections Celebal Technologies

Download Chrome Extension (See your connection in the Celebal Technologies )

chrome image
Download Now
Celebal Technologies
Celebal Technologies
www.celebaltechnologies.com

Technology Consulting and Services

Ahmedabad

201-500 Employees

39 Jobs

    Key People

  • Siddharth Ghosh

    Co-Founder & CEO

  • Vishal Kothari

    Co-Founder

RecommendedJobs for You

Tata Consultancy Services logo

Azure Data Engineer

Tata Consultancy Services

Kolkata, West Bengal, India

BayOne Solutions logo

Azure Data Engineer

BayOne Solutions

India

Black And White Business Solutions logo

Azure Data Engineer

Black And White Business Solutions

Pune

BayOne Solutions logo

Azure Data Engineer

BayOne Solutions

India

Randomtrees logo

Azure Data Engineer

Randomtrees

Chennai

SoulPage IT logo

Azure Data Engineer

SoulPage IT

Hyderabad

Addnectar Solutions logo

Azure Data Engineer

Addnectar Solutions

Chennai, Coimbatore, Mumbai (All Areas)

Viraaj HR Solutions Private Limited logo

Azure Data Engineer

Viraaj HR Solutions Private Limited

Hyderabad, Telangana, India

Viraaj HR Solutions Private Limited logo

Azure Data Engineer

Viraaj HR Solutions Private Limited

Pune, Maharashtra, India

Arcadis logo

Azure Data Engineer

Arcadis

Bengaluru