Data Engineer - Azure databricks

We are looking for a Senior Azure Data Engineer (Databricks) who can independently design, develop, and optimize cloud-based data solutions. The ideal candidate must be strong in Azure Databricks, Data Lakehouse architecture, ETL/ELT development, and data warehouse modelling. This role requires hands-on expertise in building and enhancing pipelines, migrating enterprise data (SAP/SQL Server) to cloud, and implementing scalable solutions using Azure services. This is a development-heavy role (initially 100% development, later ~60% development) with ownership of end-to-end implementation projects.

Location

Exp

Shift timing: 2-11pm

Roles & Responsibilities

• Work on Azure Databricks pipelines—enhance, modify, and optimize existing pipelines and develop new ones when required.
• Handle data migration from SAP/SQL Server and enterprise databases into Azure cloud environments (Lakehouse, Delta, Synapse).
• Configure, manage, and optimize Databricks clusters, compute, workspaces, and resources for performance and cost efficiency.
• Build scalable ETL/ELT processes using PySpark, SQL, Delta Lake, and medallion architecture across Bronze–Silver–Gold layers.
• Work extensively on Azure Data Factory, orchestration pipelines, triggers, monitoring, and integration with Databricks.
• Design and maintain data warehouse & dimensional models (Facts, Dimensions, SCD, Star/Snowflake schema).
• Ensure strong data quality, validation, governance, and security—including Unity Catalog, RBAC, and Azure platform controls.
• Develop solutions independently, involving end-to-end development, not just support—initially 100% development-focused.

Skill Set Required

• Azure Databricks (Notebooks, Clusters, Delta Lake, Optimization, Resource Configuration)
• Azure Data Lake / Lakehouse Architecture
• Azure Data Factory (Pipelines, Orchestration, Integration, Monitoring)
• Azure SQL / SQL Server
• PySpark, SQL for transformation and performance tuning
• Data Warehouse Fundamentals

o Star & Snowflake schema

o Fact & Dimension modelling

o SCD, Data Marts

• Azure Cloud Services (Key Vault, Storage, Synapse basics, RBAC, security)
• ETL/ELT development (end-to-end implementation)