Senior Data Engineer

Job Summary:

Our client is seeking a Senior Data Engineer to lead the modernization of our enterprise data platform using Microsoft Fabric, with a focus on building an AI-ready foundation. This role is key to replacing legacy ERP systems, retiring SSIS, and implementing scalable, code-driven data pipelines to support advanced analytics and future AI workloads. Our legal client is transitioning from SAP to Aderant and is currently in the first round of conversion, prioritizing 20–25 urgent integrations. The next phase will involve migrating 200–300 SAP-based reports, all within a Microsoft-based cloud environment.

Responsibilities:

• Architect and implement ETL pipelines using Microsoft Fabric (Dataflows Gen2, Pipelines, Notebooks) to support both operational reporting and AI model training.
• Migrate legacy SSIS packages to code-first, cloud-native ETL solutions.
• Design and optimize SQL Databases and Data Warehouses for structured analytics and feature engineering.
• Build and manage streaming data solutions and real-time dataflows to support predictive analytics and AI inference pipelines.
• Enable seamless data integration for ERP system replacement, ensuring clean, governed, and accessible data for downstream AI use cases.
• Collaborate with data scientists and AI engineers to ensure the platform supports model development, feature stores, and data versioning.
• Implement best practices for data lineage, metadata management, and data quality to support explainable AI and compliance.

Experience:

• Hands-on experience with Microsoft Fabric, including Dataflows Gen2, Pipelines, and Notebooks.
• Strong proficiency in SQL, ETL development, and data modeling for analytics and AI.
• Experience with streaming architectures and real-time data processing.
• Proven ability to migrate from SSIS to modern, code-based ETL frameworks.
• Familiarity with data lake house architecture, feature engineering, and AI data prep workflows.
• Understanding of AI/ML lifecycle needs, including data ingestion, transformation, and versioning.

Nice to Have:

• Experience supporting AI/ML teams with data pipelines and infrastructure.
• Familiarity with Azure Synapse, Power BI, and ML Ops tools.
• Exposure to feature stores, vector databases, or embedding pipelines.
• Knowledge of DevOps for data engineering, including CI/CD and infrastructure-as-code