Sr Data Engineer-Azure Databricks, Unity Catalog & Genie

Experience: 4-7 Years

Location: Bengaluru/Gurgaon/Kochi/Pune/Noida

Work Mode: Hybrid

Must to have: Azure Databricks, PySpark, Unity catalog, Genie

Role Overview

We are seeking an experienced Senior Developer – Azure Databricks to design, develop, and optimize enterprise-scale data and AI platforms on Azure. The ideal candidate will possess strong expertise in Azure Databricks, Unity Catalog, Delta Lake, Medallion Architecture, and hands-on experience with Databricks Genie AI for conversational analytics and AI-driven data consumption.

The role involves building modern data platforms, implementing governance frameworks, enabling self-service analytics, and supporting AI/GenAI initiatives across business functions.

Key Responsibilities

• Design and develop scalable data pipelines and ETL/ELT solutions using Azure Databricks, Apache Spark (PySpark), and Delta Lake.
• Build and optimize Lakehouse platforms following Medallion Architecture (Bronze, Silver, Gold).
• Create reusable frameworks for data ingestion, transformation, quality validation, and monitoring.
• Implement and manage Unity Catalog for enterprise data governance, metadata management, and data lineage.
• Configure catalogs, schemas, external locations, and enforce row-level and column-level security controls.
• Develop and manage Databricks Genie Spaces to enable conversational analytics and natural language querying.
• Create trusted semantic datasets, business definitions, and improve self-service analytics adoption across business teams.
• Collaborate with Data Scientists, ML Engineers, and stakeholders to deliver secure, scalable, and high-performance data solutions while leveraging Azure services such as ADF, ADLS Gen2, Key Vault, and Entra ID.

Educational Qualification: