Azure Databricks Data Engineer

Azure Databricks Data Engineer

Key Responsibilities

• ETL Development & Data Processing

  :
• Design, develop, and maintain

  ETL pipelines

  using

  Azure Databricks, PySpark, and Azure Data Factory (ADF)

  .
• Extract, transform, and load data from multiple sources into

  Azure Data Lake Storage

  .
• Optimize and fine-tune

  ETL workflows for performance and scalability

  .

• SQL Query Optimization

  :
• Write

  complex SQL queries

  , including joins, subqueries, and functions, for data processing.
• Perform

  SQL tuning and query optimization

  to enhance efficiency.
• Efficiently handle

  large datasets

  .

• Azure Cloud Integration

  :
• Implement data solutions utilizing

  Azure technologies

  such as Databricks, ADF, and ADLS.
• Work with both

  batch and streaming data processing

  for real-time analytics (preferred).
• Ensure

  data security, compliance, and reliability

  in cloud-based systems.

• Collaboration & Problem-Solving

  :
• Collaborate with cross-functional teams to support

  data analytics and business intelligence initiatives

  .
• Proactively identify and resolve

  data-related issues

  .
• Provide insights and recommendations for

  improving ETL performance

  .

Skills

• Strong experience in

  Azure Databricks, PySpark, and SQL

  .
• Proficiency in writing

  complex SQL queries

  , functions, and performing

  query tuning

  .
• Hands-on experience with

  Azure Data Factory (ADF) and Azure Data Lake Storage

  .
• Ability to handle

  large datasets efficiently

  .
• Proven experience in

  performance optimization of ETL workflows

  .
• Good understanding of

  batch and streaming data processing

  (preferred).
• Strong analytical and

  problem-solving skills

  .
• Ability to

  work independently

  and take ownership of tasks.

Qualifications

• Bachelor's degree in Information Technology, Computer Science, or a related field.