Data Engineer

Senior Data Engineer – Python & Azure

Experience:

Type:

Location:

About the Role

Senior Data Engineer

As a Senior Engineer, you won’t just build pipelines; you will partner with data scientists, analysts, and business stakeholders to ensure our data is high-quality, secure, and performant.

Key Responsibilities

• Pipeline Architecture:

  Design and maintain end-to-end ETL/ELT workflows using

  Azure Data Factory (ADF)

  ,

  Databricks

  , and

  Synapse Analytics

  .

• Data Modeling:

  Implement scalable data models (Star Schema, Snowflake, Data Vault) for both analytical and operational stores.

• Real-time & Batch Ingestion:

  Manage ingestion frameworks for diverse sources (APIs, SaaS, On-prem) and streaming use cases using

  Azure Event Hubs

  or

  Kafka

  .

• Governance & Security:

  Enforce data quality and governance standards (lineage tracking, metadata management) and manage security via

  Azure Active Directory (AAD)

  .

• Cloud Optimization:

  Manage

  ADLS Gen2

  storage and compute costs while ensuring system resilience and scalability.

• Leadership & Mentorship:

  Partner with cross-functional teams and mentor junior engineers on coding standards and Azure best practices.

Required Technical Skills

• Core Azure Services:

  Expert-level knowledge of Azure Data Factory, Azure Databricks (PySpark/Delta Lake), Synapse Analytics, and Azure Functions.

• Programming:

  Strong proficiency in

  Python

  or

  Scala

  and advanced

  SQL

  (T-SQL, Spark SQL, or Synapse SQL).

• Big Data Processing:

  Extensive experience with

  PySpark

  for distributed data processing.

• DevOps:

  Hands-on experience with

  Git

  , CI/CD (Azure DevOps/GitHub Actions), and Infrastructure as Code (Terraform/Bicep).

• Data Architecture:

  Deep understanding of Data Lakehouse architecture, partitioning, indexing, and query optimization.

Soft Skills & Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical field.
• Proven ability to work in an

  Agile

  , fast-paced environment.
• Excellent communication skills with the ability to explain technical trade-offs to non-technical stakeholders.
• A self-driven mindset with a passion for cloud technologies and problem-solving.