Data Engineer

Job Title:

Experience:

Location:

Notice Period:

Work Mode:

Mandatory Skills:

Additional Requirement:

Role Overview

Data Engineer

Key Responsibilities

• Data Pipeline Development:

  Design, build, and optimize scalable data pipelines using

  Azure Data Engineering services

  and

  Databricks

  .

• Big Data Processing:

  Develop and maintain large-scale data transformations using

  PySpark

  , ensuring performance, reliability, and efficiency.

• Data Integration:

  Integrate structured and unstructured data from various sources using SQL, Python, and Azure data services.

• Azure Platform Expertise:

  Work extensively with Azure (ADF, ADLS, Azure Databricks, Azure Functions, etc.) to enable seamless data workflows.

• GenAI Integration:

  Apply

  Generative AI techniques (minimum 3 months)

  for tasks such as data enrichment, automation, code generation, and advanced analytics.

• Data Modeling:

  Design and implement robust data models supporting both operational and analytical requirements.

• Performance Optimization:

  Improve processing time, reduce data pipeline latency, and ensure the scalability of data workloads.

• Collaboration:

  Work closely with data scientists, analysts, and business teams to deliver high-quality data solutions.

• Quality & Governance:

  Implement data quality checks, best coding practices, and governance standards across all pipelines.

• Documentation:

  Maintain clear and comprehensive technical documentation for pipelines, workflows, and engineering processes.

Required Skills & Qualifications

• 4–10 years of experience as a Data Engineer or in a similar role.
• Strong hands-on experience with

  Azure

  and

  Azure Databricks

  .
• Advanced proficiency in

  Python

  ,

  PySpark

  , and

  SQL

  .
• Experience designing efficient ETL/ELT pipelines using cloud-native tools.
• Good understanding of data warehousing concepts and distributed data systems.
• Minimum

  3 months of hands-on GenAI exposure

  (e.g., LLMs, embeddings, RAG, AI-assisted data transformations).
• Strong analytical and problem-solving skills.
• Excellent communication skills and ability to collaborate with cross-functional teams.
• Ability to work in a fast-paced environment with tight timelines.