Senior Data Engineering Consultant

Caring. Connecting. Growing together.

We are seeking a highly experienced and innovative Senior Data Engineer with deep expertise in cloud-native data engineering, particularly using Azure Databricks, Python, and AI-driven data solutions. This role will be pivotal in designing and building scalable data platforms that support advanced analytics, machine learning, and business intelligence across the organization.

Primary Responsibilities:

• Design, develop, and optimize scalable data pipelines using Databricks, Apache Spark, and Python on Azure Cloud
• Architect and implement cloud-based data solutions leveraging Azure Data Lake and Azure Data Factory
• Collaborate with data scientists and AI teams to operationalize machine learning models and integrate them into data workflows
• Build and maintain feature stores, model input pipelines, and real-time data processing frameworks
• Ensure data quality, governance, and security across all stages of the data lifecycle
• Develop reusable frameworks and automation scripts to accelerate data engineering and AI deployment
• Participate in Agile development processes, including sprint planning, code reviews, and CI/CD using GitHub or Azure DevOps
• Mentor junior engineers and contribute to knowledge sharing across teams
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regard to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• Graduate degree or equivalent experience
• 8+ years of experience in data engineering, with a solid focus on cloud platforms
• Hands-on experience with Databricks, Apache Spark, and Delta Lake
• Experience with Azure Cloud Services including Data Factory, Blob Storage
• Experience with CI/CD tools and version control systems (e.g., Git, GitHub Actions, Azure DevOps)
• Expert-level proficiency in Python for data processing, automation, and integration
• Solid understanding of ETL/ELT, data modeling, and distributed computing
• Familiarity with AI/ML concepts, model deployment, and integration into data pipelines (AI experience is a plus)
• Proficiency in SQL and working with relational and NoSQL databases
• Proven excellent problem-solving, communication, and collaboration skills

Preferred Qualifications:

• Azure certifications (e.g., Azure Data Engineer Associate, Azure AI Engineer Associate) are a plus
• Experience with MLflow, Feature Store, or other MLOps tools
• AI experience
• Exposure to Snowflake, Power BI, or other BI platforms
• Knowledge of data governance, privacy, and compliance standards