Data Engineer – Azure & Databricks

0 years

0 Lacs

Posted:1 day ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Hybrid Role: Blr/HYD/Pune/Chennai


Data Engineer


Key Responsibilities

  • Develop and optimize

    data pipelines

    using

    Azure Data Factory (ADF)

    for ingestion into

    Azure Databricks (ADB)

    .
  • Implement

    PySpark transformations

    in Databricks with a strong focus on

    performance tuning

    .
  • Build and maintain

    data layers

    (raw, curated, presentation) and ensure efficient

    export strategies

    for reporting and analytics.
  • Work with

    Azure Synapse

    for advanced querying, transformations, and reporting.
  • Configure and manage

    Azure Monitoring & Log Analytics

    for pipeline health, error handling, and automated alerts.
  • Implement automation using

    Logic Apps

    and

    Azure Functions

    for workflow orchestration and integration.
  • Ensure

    high-performance and scalable architecture

    across Databricks, Data Factory, and Synapse.
  • Collaborate with stakeholders to define best practices for

    data access, connection options, and reporting performance

    .


Required Skills

  • Strong hands-on experience with

    Azure Databricks

    (PySpark, performance tuning, data modeling).
  • Proficiency in

    Azure Data Factory (ADF)

    – pipeline development, monitoring, troubleshooting.
  • Solid understanding of

    Azure Synapse Analytics

    (SQL Pools, performance optimization).
  • Experience with

    Azure Monitoring, Log Analytics, and error-handling frameworks

    .
  • Good knowledge of

    Logic Apps & Azure Functions

    for automation and orchestration.
  • Strong understanding of

    data architecture patterns

    (data lakes, data warehouses, reporting layers).
  • Excellent troubleshooting, performance optimization, and analytical skills.


Good to Have

  • Experience with

    CI/CD for data pipelines

    (Azure DevOps).
  • Knowledge of

    Delta Lake and Lakehouse architecture

    .
  • Exposure to BFSI/Insurance domain data solutions.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now

RecommendedJobs for You