Lead Data Engineer

You are a Senior Data Engineer with expertise in constructing scalable data pipelines utilizing Microsoft Fabric. Your primary responsibilities will involve developing and managing data pipelines through Microsoft Fabric Data Factory and OneLake. You will be tasked with designing and creating ingestion and transformation pipelines for both structured and unstructured data. It will be your responsibility to establish frameworks for metadata tagging, version control, and batch tracking to ensure the security, quality, and compliance of data pipelines. Additionally, you will play a crucial role in contributing to CI/CD integration, observability, and documentation. Collaboration with data architects and analysts will be essential to align with business requirements effectively. To qualify for this role, you should possess at least 6 years of experience in data engineering, with a minimum of 2 years of hands-on experience working on Microsoft Fabric or Azure Data services. Proficiency in tools like Azure Data Factory, Fabric, Databricks, or Synapse is required. Strong SQL and data processing skills (such as PySpark and Python) are essential. Previous experience with data cataloging, lineage, and governance frameworks will be beneficial for this position.,