Etl Developer

Role & responsibilities

• Design, build, and maintain scalable

  ETL pipelines

  to ingest and transform data from diverse sources.
• Work extensively with

  Azure Data Lake

  ,

  Azure Data Factory (ADF)

  , and related Azure services for data integration and orchestration.
• Extract data from various structured and unstructured sources using

  APIs

  , flat files, databases, or cloud endpoints.
• Implement data flows to securely move and transform data into

  Azure Data Lake

  storage layers.
• Write, optimize, and maintain complex

  SQL queries

  for data transformation and analysis.
• Ensure high data quality, integrity, and reliability throughout the data pipeline.
  
• Collaborate with data analysts, BI developers, and stakeholders to understand data requirements and deliver robust solutions.
  
• Document ETL processes, data mapping, and data lineage for transparency and maintenance.
  
• Monitor, troubleshoot, and optimize ETL pipelines for performance and cost efficiency.
  

Preferred candidate profile

• 3+ years

  of hands-on experience in

  ETL development

  and

  data engineering

  .
• Strong expertise in

  Azure Data Lake

  ,

  Azure Data Factory (ADF)

  , and pipeline creation.
• Proficiency in

  SQL

  and

  T-SQL

  for data manipulation and transformation.
• Solid understanding of

  ETL principles

  and the

  data lifecycle

  (ingestion to storage).
• Experience working with

  REST APIs

  or other APIs to extract and load data from external sources.
• Clear understanding of the differences between a

  Data Lake

  and a

  Data Warehouse

  .
• Familiarity with

  data governance

  ,

  metadata management

  , and

  data quality

  standards.
• Experience with

  JSON/XML data structures

  ,

  flat files

  , and cloud-native storage.
• Strong analytical, debugging, and problem-solving skills.
  

Nice to Have:

• Experience with

  Power BI

  or other BI/visualization tools.
• Knowledge of

  Azure Synapse

  ,

  Databricks

  , or

  Data Bricks Notebooks

  .
• Basic scripting in

  Python

  or

  PySpark

  .
• Exposure to

  CI/CD

  pipelines using

  Azure DevOps

  or Git.