Azure ADB +Unity Catalog

Swift Staffing

7 - 10 years

5 - 10 Lacs

Hyderabad Bengaluru

Posted:1 month ago| Platform:

Apply

Skills Required

Azure Data Engineering Azure Data Factory Azure Databricks Catalog

Work Mode

Work from Office

Job Type

Full Time

Job Description

Role & responsibilities

Responsibilities:

Design, build, and optimize large-scale data pipelines using Azure Data Factory, Azure Databricks, and Delta Lake
Lead the migration of legacy Hive-metastore-based Databricks workspaces to Unity Catalog
Implement data governance and security models using Unity Catalog (external locations, catalogs, schemas, access controls, and tags)
Develop and manage ETL/ELT pipelines for structured and semi-structured data across Azure Data Lake Storage (ADLS)
Collaborate with data architects, analysts, and stakeholders to deliver clean, accessible, and governed datasets
Optimize Databricks jobs for performance and cost-efficiency, leveraging cluster configurations and job orchestration
Apply metadata management, tagging, and data lineage practices for cataloged assets
Monitor and troubleshoot data pipelines, ensuring data quality, consistency, and observability

Enforce DevOps best practices using CI/CD pipelines with Git, Databricks Repos, and Infrastructure as Code (IaC).

Required Skills:

7+ years of experience in data engineering on the Azure ecosystem
Expert-level skills in Azure Databricks, Delta Lake, and PySpark
Strong understanding of Unity Catalog architecture, permissions model, and table/volume management
Experience migrating data platforms to Unity Catalog (including table remapping, external locations, access control setup)
Proficient in Azure Data Factory, Azure Data Lake Storage (Gen2), and Azure SQL / Synapse
Hands-on with Databricks REST APIs, orchestration workflows, and cluster management
Familiarity with data governance, RBAC, ABAC, and sensitivity labeling
Solid understanding of version control (Git), DevOps tools, and CI/CD pipelines

More Jobs at Swift Staffing

International Voice Process Executive

Thane

1 - 5 yrs

INR 4 - 4 Lacs

Azure Network Engineer

Hyderabad, Chennai, Bengaluru

3 - 5 yrs

INR 4 - 7 Lacs

Autodesk (Civil3D) Full Stack Developer

Chennai

3 - 8 yrs

INR 5 - 9 Lacs

Azure Devops Engineer

Hyderabad

3 - 5 yrs

INR 3 - 7 Lacs

Dot net with Angular developer

Bengaluru

3 - 6 yrs

INR 6 - 12 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.