Lead I - Data Engineering

Overview

Overview We are seeking a highly skilled Senior Data Engineer with 5+ years of experience in building scalable data solutions on the Azure ecosystem. The ideal candidate will have strong hands-on expertise in Databricks, Spark (Java), modern data lake architectures, and end-to-end ETL development.

Mandatory Skills: Azure Cloud,Java,Pyspark

Key Responsibilities

• Design, develop, and maintain

  scalable data pipelines

  using Azure services and Databricks.
• Build efficient

  data ingestion and transformation workflows

  using Databricks with

  PySpark

  and

  Delta Lake

  .
• Develop and optimize ETL pipelines using

  Java

  and

  SQL

  .
• Implement best practices for

  data modelling

  ,

  data quality

  ,

  governance

  , and

  pipeline performance

  .
• Work with relational and lakehouse databases including

  PostgreSQL

  and

  Delta Lake

  .
• Collaborate with cross-functional teams to support

  production workloads

  , including troubleshooting and performance tuning.

Required Skills & Experience

• 5+ years of hands-on

  data engineering

  experience.
• Strong working knowledge of the

  Azure ecosystem

  , particularly:
• Azure Databricks
• Azure Data Lake Storage (ADLS)
• Azure Functions
• Azure Data Factory (preferred)
• Expertise in

  Apache Spark

  with

  Java

  and

  PySpark

  .
• Advanced

  SQL

  skills, including query optimization.
• Strong understanding of

  ETL/ELT methodologies

  and

  data warehousing concepts

  .
• Experience with

  Delta Lake

  (schema evolution, ACID transactions, time travel).
• Experience working with

  PostgreSQL

  or similar relational databases.
• Strong analytical, problem-solving, and communication skills.
• Ability to work proactively in a fast-paced environment.

Preferred Skills

• Experience with

  CI/CD

  tools such as Azure DevOps and Git.
• Knowledge of

  Infrastructure as Code (IaC)

  tools such as

  Terraform

  .
• Exposure to streaming platforms like

  Kafka

  or

  Azure Event Hub

  .
• Familiarity with

  data governance

  solutions such as

  Azure Purview

  .