Analyst - Data Engineering

About the Role:

We are seeking a skilled and experienced Data Engineer to join our team and help build,

optimize, and maintain data pipelines and architectures. The ideal candidate will have deep

expertise in the Microsoft data engineering ecosystem, particularly leveraging tools such as

Azure Data Factory, Databricks, Alteryx, Synapse Analytics, Microsoft Fabric, and a

strong command of SQL, Python, and Apache Spark.

Key Responsibilities:

? Design, develop, and optimize scalable data pipelines and workflows using Azure

Data Factory, Synapse Pipelines, and Microsoft Fabric.

? Responsible for transforming and migrating existing Alteryx workflows into scalable

Azure Databricks pipelines, ensuring optimized performance and maintainability.

? Collaborate with data engineering and analytics teams to redesign Alteryx pipelines

into Spark-based solutions on Azure Databricks, leveraging Delta Lake and Azure

services for automation and efficiency.

? Build and maintain ETL/ELT processes for ingesting structured and unstructured data

from various sources.

? Develop and manage data transformation logic using Databricks (PySpark/Spark

SQL) and Python.

? Collaborate with data analysts, architects, and business stakeholders to understand

requirements and deliver high-quality data solutions.

? Ensure data quality, integrity, and governance across the data lifecycle.

? Implement monitoring and alerting for data pipelines to ensure reliability and

performance.

? Work with Azure Synapse Analytics to build data models and enable analytics and

reporting.

? Utilize SQL for querying and managing large datasets efficiently.

? Participate in data architecture discussions and contribute to technical design

decisions.

Required Skills and Qualifications:

? 3+ years of experience in data engineering or a related field.

? Strong experience in Alteryx workflows and data preparation/ETL processes.

? Strong proficiency in the Microsoft Azure data ecosystem including:

o Azure Data Factory (ADF)

o Azure Synapse Analytics

o Microsoft Fabric

o Azure Databricks

? Solid experience with Python and Apache Spark (including PySpark).

? Advanced skills in SQL for data manipulation and transformation.

? Experience in designing and implementing data lakes and data warehouses.

? Familiarity with data governance, security, and compliance standards.

? Strong analytical and problem-solving skills.

? Excellent communication and collaboration abilities.

Preferred Qualifications:

? Microsoft Azure certifications (e.g., Azure Data Engineer Associate).

? Experience with DevOps tools and CI/CD practices in data workflows.

? Knowledge of REST APIs and integration techniques.

? Background in agile methodologies and working in cross-functional teams.