Jobs
Interviews
PwC India logo

Senior Associate & Manager - Databricks | Azure | PySpark - Kolkata | Bengaluru | Hyderabad

6 years

0 Lacs

Kolkata metropolitan area West Bengal India

Posted:3 days ago| Platform: Linkedin logo

Apply

Skills Required

databricks azure pyspark data sql support design apache integration etl spark extract analyze governance security leadership mentoring analysis storage git devops communication developer kafka scripting python agile scrum

Work Mode

On-site

Job Type

Full Time

Job Description

Job Title:

Senior Data Engineer – Databricks | Azure | PySpark

Location:

Experience:

Job Type:

Industry:

 

Job Summary:

Senior Associate

 

Key Responsibilities:

  • Design, build, and maintain scalable data pipelines using

    Apache Spark

    and

    PySpark

    in

    Databricks

    .
  • Develop and manage data integration and ETL workflows in

    Azure Data Factory

    and Databricks.
  • Optimize and troubleshoot Spark jobs and ensure efficient use of compute and memory resources.
  • Write complex SQL queries to extract, transform, and analyze large datasets across multiple data sources.
  • Implement data governance, quality, and security practices as per organizational standards.
  • Collaborate with cross-functional teams to define data requirements and implement scalable solutions.
  • Monitor, maintain, and optimize performance of data platforms hosted on

    Azure

    (e.g., ADLS, Azure Synapse, Azure SQL).
  • Provide technical leadership and mentoring to junior team members.

 

Required Skills & Qualifications:

  • 6+ years

    of hands-on experience in

    Data Engineering

    .
  • Strong expertise in

    Databricks

    including notebooks, clusters, Delta Lake, and job orchestration.
  • Proficient in

    PySpark

    and distributed computing using Apache Spark.
  • Expert-level knowledge of

    SQL

    for data analysis, transformation, and optimization.
  • Extensive experience with

    Azure Cloud Services

    including

    Azure Data Factory, Azure Data Lake Storage (ADLS)

    ,

    Azure Synapse

    , and

    Azure SQL DB

    .
  • Experience with

    CI/CD pipelines

    , version control (e.g., Git), and DevOps best practices in Azure environment.
  • Solid understanding of

    data modeling

    ,

    data warehousing

    , and

    data governance

    practices.
  • Strong analytical and problem-solving skills.
  • Strong communication skills and end client facing experience
 

Preferred Qualifications:

  • Databricks Certified Developer or Microsoft Azure certification(s) (e.g., DP-203).
  • Experience with real-time data streaming (e.g., Kafka, Azure Event Hubs) is a plus.
  • Familiarity with scripting languages such as Python for automation.
  • Experience working in Agile/Scrum environments.
  • Insurance domain knowledge is a plus.

 

 

More Jobs at PwC India

IN-Associate_ IRDA-Debt_ Advisory_Hyderabad

Hyderabad, Telangana, India

1 - 2 yrs
|

Salary: Not disclosed

IN_Associate_ GTM_Strategy&_ Advisory_Mumbai

Mumbai, Maharashtra, India

Experience: Not specified

|

Salary: Not disclosed

IN- Associate – SAP MM – SAP-Advisory - Mumbai

Mumbai, Maharashtra, India

5 - 8 yrs
|

Salary: Not disclosed

IN_Senior Associate__Full Stack Developer-- Data and Analytics_Advisory_Hyderabad

Hyderabad, Telangana, India

4 - 7 yrs
|

Salary: Not disclosed

IN-Associate_SAP TM_SAP_Advisory_Kolkata

Greater Kolkata Area

5 - 8 yrs
|

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Google Play App Store
coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
PwC India logo
PwC India

Business Consulting and Services

Kolkata West Bengal

RecommendedJobs for You

PwC India logo

Senior Associate & Manager - Databricks | Azure | PySpark - Kolkata | Bengaluru | Hyderabad

PwC India

Kolkata metropolitan area, West Bengal, India

PwC India logo

Senior Associate & Manager - Databricks | Azure | PySpark - Kolkata | Bengaluru | Hyderabad

PwC India

Kolkata metropolitan area, West Bengal, India