Lead I - Data Engineering

5 years

0 Lacs

Posted:2 days ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Role Description

Job Title: Senior Data Engineer – Azure & Databricks

Overview

We are seeking a highly skilled

Senior Data Engineer

with

5+ years of hands-on experience

in building scalable, high-performance data solutions within the

Azure ecosystem

. The ideal candidate will have strong expertise in

Databricks, Apache Spark (Java), modern data lake architectures

, and end-to-end ETL pipeline development. This role involves designing, developing, and optimizing data systems that support business-critical analytics and operational use cases.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure services and Databricks.
  • Build efficient data ingestion and transformation workflows leveraging Databricks, Spark, and Delta Lake.
  • Develop, enhance, and optimize ETL pipelines using Java, Spark, and SQL.
  • Implement industry-best practices for data modeling, data quality, governance, and pipeline performance.
  • Work with relational and lakehouse data platforms, including PostgreSQL and Delta Lake.
  • Collaborate with cross-functional teams to ensure smooth execution, deployment, and support of production data workloads.
  • Troubleshoot performance issues and optimize workloads for reliability and scalability.

Required Skills & Experience

  • 5+ years of hands-on experience in Data Engineering.
  • Strong working knowledge of the Azure ecosystem, specifically:
    • Azure Databricks
    • Azure Data Lake Storage (ADLS)
    • Azure Functions
    • Azure Data Factory (preferred but not mandatory)
  • Expertise in Apache Spark with Java (Spark Core, Spark SQL, Spark Streaming).
  • Advanced SQL skills including performance tuning and complex query optimization.
  • Strong understanding of ETL/ELT methodologies, data warehousing concepts, and lakehouse architectures.
  • Experience with Delta Lake features such as schema evolution, ACID transactions, and time travel.
  • Hands-on experience with PostgreSQL or similar relational databases.
  • Excellent analytical, problem-solving, and debugging skills.
  • Strong communication skills with the ability to work with technical and non-technical stakeholders.
  • Self-motivated, proactive, and comfortable working in fast-paced environments.

Preferred Skills

  • Experience with CI/CD tools such as Azure DevOps and Git.
  • Knowledge of Infrastructure-as-Code (IaC) tools like Terraform.
  • Exposure to streaming technologies such as Kafka or Azure Event Hub.
  • Familiarity with data governance tools such as Azure Purview.

Skills

Azure Cloud,Java,Pyspark

Mock Interview

Practice Video Interview with JobPe AI

Start Java Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Java Skills

Practice Java coding challenges to boost your skills

Start Practicing Java Now
UST logo
UST

IT Services and IT Consulting

Aliso Viejo CA

RecommendedJobs for You

pune, maharashtra, india

pune, maharashtra, india

pune, thiruvananthapuram