ADF- Azure Data Engineer

• Design, build, and maintain scalable and reliable data pipelines using tools like Spark, Python, or SQL.
• Develop ETL/ELT processes to ingest data from various sources (structured/unstructured) into data lakes/warehouses.
• Collaborate with stakeholders to gather requirements and implement efficient data models.
• Ensure data quality, consistency, and governance across different data platforms.
• Optimize data workflows and performance tuning of ETL jobs.
• Work on cloud data engineering technologies like AWS, Azure, or GCP.
• Implement monitoring and logging for data processes and pipelines.
• Automate repetitive data engineering tasks and support CI/CD pipelines.
• Collaborate with cross-functional teams to support data-driven initiatives.

Required Skills:

• Proficiency in

  SQL

  ,

  Python

  , and

  Spark/PySpark

  .
• Experience with

  ETL tools

  like Informatica, Talend, Apache NiFi, or custom scripting.
• Strong understanding of

  data warehousing

  concepts and

  data modeling

  (Star/Snowflake schema).
• Experience with

  cloud platforms

  : AWS (Glue, Redshift, S3), Azure (ADF, Synapse), or GCP (BigQuery, Dataflow).
• Familiarity with

  version control

  tools (e.g., Git) and

  CI/CD pipelines

  .
• Knowledge of

  data governance

  ,

  security

  , and

  compliance

  practices.