Lead I - Data Engineering

5 years

0 Lacs

Posted:1 day ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Role Description

enior Data Engineer (Azure/Databricks)

Overview

We are seeking a

Senior Data Engineer

with

5+ years of hands-on experience

in building and scaling data solutions within the

Azure ecosystem

. The ideal candidate will have deep expertise in

Databricks

,

PySpark

, and

modern data lake architectures

, and be adept at designing robust

ETL pipelines

and data workflows.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure and Databricks.
  • Build efficient data ingestion and transformation workflows with PySpark and Delta Lake.
  • Develop, optimize, and maintain ETL/ELT pipelines using Python and SQL.
  • Implement data modeling, data quality, and governance best practices.
  • Work with relational databases (e.g., PostgreSQL) and lakehouse architectures (e.g., Delta Lake).
  • Collaborate with cross-functional teams to troubleshoot, optimize, and support production workloads.

Required Skills & Experience

  • 5+ years of data engineering experience.
  • Strong proficiency in the Azure ecosystem, including:
    • Azure Databricks
    • Azure Data Lake Storage (ADLS)
    • Azure Functions
    • Azure Data Factory (preferred)
  • Advanced PySpark and SQL expertise, including performance tuning.
  • Deep understanding of ETL/ELT design patterns and data warehousing concepts.
  • Experience with Delta Lake (ACID transactions, schema evolution, time travel).
  • Hands-on experience with PostgreSQL or similar RDBMS.
  • Strong analytical, problem-solving, and communication skills.
  • Self-motivated and comfortable in fast-paced, agile environments.

Preferred Skills

  • Familiarity with CI/CD pipelines (Azure DevOps, Git).
  • Knowledge of Infrastructure-as-Code tools (Terraform).
  • Exposure to real-time streaming technologies (Kafka, Event Hub).
  • Awareness of data governance tools (e.g., Microsoft Purview).

Technical Stack

Languages & Tools:

Python, PySpark, SQL

Platforms:

Azure Databricks, ADLS, Azure Functions, ADF

Databases:

PostgreSQL, Delta Lake

DevOps & Governance:

Azure DevOps, Git, Terraform, PurviewWould you like me to turn this into a

forma

Skills

Python,Spark,Azure

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
UST logo
UST

IT Services and IT Consulting

Aliso Viejo CA

RecommendedJobs for You

pune, maharashtra, india

pune, thiruvananthapuram