Senior Data Engineer - Databricks, Azure & Mosica AI

Job Title: Senior Data Engineer Databricks, Azure & Mosaic AI Role Summary: We are seeking a Senior Data Engineer with extensive expertise in Data & Analytics platform modernization using Databricks, Azure, and Mosaic AI. This role will focus on designing and optimizing cloud-based data architectures, leveraging AI-driven automation to enhance data pipelines, governance, and processing at scale. Key Responsibilities: Architect & modernize Data & Analytics platforms using Databricks on Azure. Design and optimize Lakehouse architectures integrating Azure Data Lake, Databricks Delta Lake, and Synapse Analytics. Implement Mosaic AI for AI-driven automation, predictive analytics, and intelligent data engineering solutions. Lead the migration of legacy data platforms to a modern cloud-native Data & AI ecosystem. Develop high-performance ETL pipelines, integrating Databricks with Azure services such as Data Factory, Synapse, and Purview. Utilize MLflow & Mosaic AI for AI-enhanced data processing and decision-making. Establish data governance, security, lineage tracking, and metadata management across modern data platforms. Work collaboratively with business leaders, data scientists, and engineers to drive innovation. Stay at the forefront of emerging trends in AI-powered data engineering and modernization strategies. Required Skills & Qualifications: Experience in Data Engineering, Cloud Platforms, and AI-driven automation. Expertise in Databricks (Apache Spark, Delta Lake, MLflow) and Azure (Data Lake, Synapse, ADF, Purview). Strong experience with Mosaic AI for AI-powered data engineering and automation. Advanced proficiency in SQL, Python, and Scala for big data processing. Experience in modernizing Data & Analytics platforms, migrating from on-prem to cloud. Knowledge of Data Lineage, Observability, and AI-driven Data Governance frameworks. Familiarity with Vector Databases & Retrieval-Augmented Generation (RAG) architectures for AI-powered data analytics. Strong leadership, problem-solving, and stakeholder management skills. Preferred Skills: Experience with Knowledge Graphs (Neo4J, TigerGraph) for data structuring. Exposure to Kubernetes, Terraform, and CI/CD for scalable cloud deployments. Background in streaming technologies (Kafka, Spark Streaming, Kinesis).