Data Engineer

Job Title:

Data Engineer

Experience Range:

6-9 years

Location

Notice period

Job Summary:

Data Engineer

Key Responsibilities:

For External Candidates:

• Data Pipeline Development:

  
  Build and optimize data pipelines to ingest, transform, and load data from multiple sources using Azure Databricks, Snowflake, and DBT.

• Data Modeling & Architecture:

  
  Design efficient data models and structures within Snowflake ensuring optimal performance and accessibility.

• Data Transformation:

  
  Implement standardized and reusable data transformations in DBT for reliable analytics and reporting.

• Performance Optimization:

  
  Monitor and tune data workflows for performance, scalability, and fault-tolerance.

• Cross-Team Collaboration:

  
  Partner with data scientists, analysts, and business users to support analytics and machine learning projects with reliable, well-structured datasets.

Additional Responsibilities (Internal Candidates):

• Implement and manage

  CI/CD pipelines

  using tools such as

  Jenkins

  ,

  Azure DevOps

  , or

  GitHub

  .
• Develop

  data lake solutions

  using

  Scala

  and

  Python

  in a

  Hadoop/Spark

  ecosystem.
• Work with

  Azure Data Factory

  and orchestration tools to schedule, monitor, and maintain workflows.
• Apply deep understanding of

  Hadoop architecture

  , Spark, Hive, and storage optimization.

Mandatory Skills:

• Hands-on experience with:
  

• Azure Databricks

  (data processing and orchestration)

• Snowflake

  (data warehousing)

• DBT

  (data transformation)

• Azure Data Factory

  (pipeline orchestration)
• Strong

  SQL

  and

  data modeling

  capabilities
• Proficiency in

  Scala

  and

  Python

  for data engineering use cases
• Experience with

  big data ecosystems

  : Hadoop, Spark, Hive
• Knowledge of

  CI/CD pipelines

  (Jenkins, GitHub, Azure DevOps)

Qualifications:

• Bachelors degree in

  Computer Science

  ,

  Data Engineering

  , or a related field
• 6–9 years of relevant experience in

  data engineering

  or

  data infrastructure

  roles