Data Engineer

Job Summary:

Key Responsibilities:

• Data Pipeline Development: Design, build, and maintain scalable and efficient data pipelines and ETL/ELT workflows using Azure Data Factory (ADF) and Microsoft Fabric Pipelines to ingest, transform, and load data from diverse sources.
• Microsoft Fabric Implementation: Implement and manage data solutions within the Microsoft Fabric ecosystem, including Lakehouse and Data Warehouse architectures, using tools like Synapse Data Engineering notebooks (Data modeling, SQL, and ETL)
• Data Modeling & Architecture: Design and develop efficient data architecture and dimensional data models (e.g., star schemas) to support enterprise reporting and Business Intelligence (BI) needs.
• Power BI Integration: Collaborate with BI developers and analysts to deliver high-quality Power BI datasets and semantic models. Develop and optimize interactive reports and dashboards with complex DAX calculations and Power Query transformations.
• Data Quality & Governance: Implement data validation, cleansing procedures, and robust data governance practices, including role-level security (RLS), access control, and lineage tracking (using tools like Microsoft Purview).
• Performance Optimization: Continuously monitor and fine-tune data pipelines, queries (SQL and PySpark), and data storage solutions for performance, scalability, and cost efficiency.
• Collaboration & Documentation: Work closely with business stakeholders, data scientists, and analysts to understand requirements and translate them into technical solutions. Create and maintain clear documentation for data models, processes, and configurations.
• Automation & CI/CD: Automate deployments and monitoring using CI/CD pipelines with tools like Azure DevOps or GitHub for data solutions.

Required Skills & Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Proven experience (e.g., 4-5+ years) as a Data Engineer, preferably on Microsoft Azure and Fabric platforms.
• Strong proficiency in SQL, Python/PySpark, and Kusto Query Language (KQL).
• Hands-on experience with Microsoft Azure Data Factory and the broader Microsoft Fabric ecosystem.
• Expertise in Power BI, including data modeling, DAX, and creating interactive visualizations.
• Solid understanding of data warehousing concepts, ETL/ELT processes, and lakehouse architectures.
• Familiarity with data governance, security, and compliance best practices.
• Strong problem-solving, debugging, and performance tuning skills.

Preferred Qualifications:

• Microsoft certifications such as the Microsoft Certified: Fabric Analytics Engineer Associate or Microsoft Certified: Fabric Data Engineer Associate are highly desirable.
• Experience with Azure services like Azure Synapse Analytics, Azure Databricks, and Azure Data Lake Storage Gen2.
• Knowledge of real-time data streaming concepts (e.g., Kafka or Azure Event Hub).