Azure Data Engineering

Key Skills:

Roles and Responsibilities:

• Design and implement scalable data pipelines and ETL/ELT processes using

  Azure Data Factory

• Develop and optimize relational and non-relational databases for performance, scalability, and reliability
• Build and manage data lakes and data warehouses using

  Azure Synapse Analytics

  ,

  Data Lake Storage

  , and

  SQL-based solutions

• Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions
• Ensure data governance, security, and compliance with industry standards
• Lead architectural discussions and mentor junior engineers
• Utilize strong proficiency in

  SQL

  and database design including

  Azure SQL

  ,

  PostgreSQL

  ,

  Oracle

  ,

  MongoDB

  , and

  CosmosDB

• Apply deep understanding of

  data modeling

  ,

  indexing

  , and

  performance tuning

• Leverage hands-on experience with Azure services such as:
• Azure Data Factory
• Synapse Analytics
• Data Lake Storage
• Azure SQL
• Azure Functions
• Azure Databricks
• Power BI
• Employ proficiency in

  Python

  and

  PySpark

  for data processing
• Manage large database migrations to Azure using

  Database Migration tools

  and

  Azure Data Studio

• Utilize

  Airflow

  for workflow management
• Work with big data tools including

  Spark

  ,

  Kafka

  , and

  Delta Lake

• Familiarity with

  CI/CD pipelines

  and

  infrastructure-as-code

  using

  Terraform

  and

  ARM templates

• Demonstrate excellent problem-solving and communication skills

Skills Required:

• Hands-on expertise in Azure Data Engineering services
• Strong SQL and database development background
• Proficient in Python and PySpark
• Experience with big data and streaming tools (Kafka, Spark)
• Knowledge of data governance and compliance frameworks
• Ability to manage migrations and optimize data architecture
• Strong communication, mentoring, and problem-solving abilities

Education: