Azure Data Engineer

Azure Data Engineer

Key Responsibilities:

• Design and develop advanced ETL pipelines for data ingestion and egress for batch data.
• Build scalable data solutions using

  Azure Data Factory (ADF)

  ,

  Databricks

  ,

  Spark (PySpark & Scala Spark)

  , and other Azure services.
• Troubleshoot data jobs, identify issues, and implement effective root cause solutions.
• Collaborate with stakeholders to gather requirements and propose efficient solution designs.
• Ensure data quality, reliability, and adherence to best practices in data engineering.
• Maintain detailed documentation of problem definitions, solutions, and architecture.
• Work independently with minimal supervision while ensuring project deadlines are met.

Required Skills & Qualifications:

• Microsoft Certified: Azure Fundamentals

  (preferred).

• Microsoft Certified: Azure Data Engineer Associate

  (preferred).
• Proficiency in

  SQL

  ,

  Python

  , and

  Scala

  .
• Strong knowledge of

  Azure Cloud services

  ,

  ADF

  , and

  Databricks

  .
• Hands-on experience with

  Apache Spark

  (PySpark & Scala Spark).
• Expertise in designing and implementing complex ETL pipelines for batch data.
• Strong troubleshooting skills with the ability to perform root cause analysis.

Soft Skills:

• Excellent verbal and written communication skills.
• Strong documentation skills for drafting problem definitions and solutions.
• Ability to effectively gather requirements and propose solution designs.
• Self-driven with the ability to work independently with minimal supervision.