Senior Data Engineer

Role & responsibilities

Preferred candidate profile

Job Title: Senior Data Engineer - Azure Stack

Senior Data Engineer

Key Responsibilities:

• Data Engineering:

  Design and implement scalable data pipelines, ETL workflows, and processing frameworks within Azure Stack and hybrid cloud environments.

• Cloud Architecture:

  Leverage the capabilities of

  Azure Stack

  ,

  Azure Data Factory

  ,

  Azure Synapse Analytics

  , and other Azure cloud services to integrate, manage, and optimize enterprise data solutions.

• Data Integration & Storage:

  Build and optimize data lakes, data warehouses, and cloud-based storage systems on Azure, integrating structured and unstructured data from multiple sources.

• Performance Optimization:

  Continuously improve data pipelines for performance, scalability, and cost-efficiency. Implement automation and monitoring solutions to ensure smooth operation.

• Cross-Functional Collaboration:

  Work closely with product managers, data scientists, and business intelligence teams to implement data models, data visualizations, and analytics solutions.

• Mentorship:

  Provide mentorship and technical leadership to junior engineers, helping them improve their skills in cloud data technologies, coding standards, and engineering best practices.

• Cloud Migration:

  Assist in the migration of on-premises data infrastructure to

  Azure Stack

  , working closely with infrastructure teams to ensure a seamless transition.

Required Skills & Qualifications:

• Experience:

  7+ years of experience in data engineering, with at least 2+ years of experience working specifically with

  Azure Stack

  and

  Azure cloud

  services in a hybrid environment.

• Azure Stack Expertise:

  Proficient in

  Azure Stack Hub

  ,

  Azure Data Factory

  ,

  Azure SQL Database

  ,

  Azure Synapse Analytics

  ,

  Azure Databricks

  , and

  Azure Blob Storage

  .

• Programming Skills:

  Proficiency in

  SQL

  ,

  Python

  , or similar scripting languages for building data pipelines and processing large datasets.

• ETL Tools:

  Experience with building and optimizing ETL workflows using

  Azure Data Factory

  ,

  Apache Airflow

  , or other similar tools.

• Data Integration:

  Experience working with data integration tools, including both structured and unstructured data sources, and technologies such as

  Kafka

  ,

  Spark

  ,

  Hadoop

  , or others.

• Data Warehousing & Storage:

  Strong understanding of

  data warehousing

  concepts (e.g., star/snowflake schemas) and expertise with

  Azure Data Lakes

  and

  SQL/NoSQL storage solutions

  .

• Big Data Technologies:

  Familiarity with big data technologies like

  Apache Hadoop

  ,

  Apache Spark

  ,

  Databricks

  , and

  Kafka

  is a plus.

Soft Skills:

• Excellent analytical and problem-solving abilities.
• Strong communication skills to articulate technical details to non-technical stakeholders.
• Ability to work collaboratively in a distributed team environment.
• Self-motivated, proactive, and able to work under minimal supervision.
• Strong attention to detail, ensuring data quality and security.