Talentmatics logo

Data Engineer – Databricks

Talentmatics

6 - 8 years

Pune

Posted:6 days ago

Job Description

Data Engineer – Databricks


Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, and SQL.
  • Build and manage data ingestion and transformation processes using Delta Lake and Delta Tables.
  • Implement Auto Loader (cloudFiles), schema inference, and schema evolution.
  • Develop and manage Spark Declarative Pipelines (SDP/DLT).
  • Configure and manage Databricks Workflows, Jobs, Tasks, and Triggers.
  • Implement data governance, security, and access management using Unity Catalog.
  • Design dimensional data models, data marts, and Slowly Changing Dimensions (SCD).
  • Develop and optimize Databricks SQL Dashboards and work with AI/BI, Genie, or similar BI tools.
  • Monitor and troubleshoot data pipelines to ensure performance, reliability, and scalability.


Requirements

  • 6–8 years of overall experience in Data Engineering.
  • Strong hands-on experience with Azure Databricks, PySpark, and SQL.
  • Expertise in Delta Lake and Delta Tables.
  • Practical experience with Auto Loader (cloudFiles), schema inference, and schema evolution.
  • Hands-on experience with Spark Declarative Pipelines (SDP/DLT).
  • Good understanding of Databricks Workflows, Jobs, Tasks, and Triggers.
  • Strong knowledge of Unity Catalog, data governance, and access management.
  • Experience with dimensional modelling, data marts, and SCD.
  • Familiarity with Databricks SQL Dashboards, AI/BI, and Genie.
  • Working knowledge of cloud platforms such as Azure, AWS, or GCP.


Qualifications

  • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
  • Strong understanding of data engineering concepts, ETL/ELT processes, and data pipeline architecture.
  • Excellent analytical, problem-solving, and troubleshooting skills.
  • Ability to work independently and collaborate effectively with cross-functional teams.


Skills Required

Data Engineering Azure Databricks Pyspark SQL Data Processing Data Ingestion Spark Data Governance Access Management BI Tools Cloud Platforms Azure

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Talentmatics logo
Talentmatics

RecommendedJobs for You

Talentmatics logo
Talentmatics logo