Databricks Technical Lead

8 - 12 years

Bengaluru

Posted:7 hours ago

Job Description

Data Engineer - Azure Databricks

Experience:8 Years to 12 Years (5+ Years of Relevant)

Work Mode:Hybrid

Location:Bangalore

Notice Period:Immediate to 30 Days Max only

Key Responsibilities

  • ETL Development: Build, test, and deploy ETL pipelines in Azure Databricks using PySpark, Spark SQL, and Delta Live Tables.
  • Migration Execution: Rewrite and optimize existing AWS Databricks in notebooks and jobs for Azure Databricks.
  • Data Ingestion: Build ingestion frameworks to load data from various sources into ADLS Gen2 and Delta tables.
  • Performance Tuning: Optimize Spark jobs for cost and performance (partitioning, Z-Ordering, caching, cluster sizing).
  • Orchestration: Manage job scheduling and dependencies using Databricks Workflows and Azure Data Factory.

Required Skills & Experience

  • Data Engineering: 8+ years in data engineering, with at least 34 years in Azure Databricks.
  • ETL Development: Strong experience building ETL pipelines with PySpark and Spark SQL. Experience with Delta Lake and Delta Live Tables.
  • Programming: Proficiency in Python and SQL.
  • Migration: Experience migrating Databricks workloads from AWS to Azure is a plus.

Skills Required

Azure Databricks Python Data Engineering Azure Data Factory Engineering Microsoft Azure SQL Technical

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Work from Office

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Genpact logo
Genpact

Business Consulting and Services

New York NY

RecommendedJobs for You