Job
Description
Senior Consultant - Platform Engineer Responsibilities • Design, build, and optimize platform solutions in Azure Databricks to support scalable, high-performance, and secure data engineering workflows. • Implement and maintain platform automation for infrastructure, monitoring, and deployment pipelines using modern DevOps practices. • Align with best practices for platform architecture, governance, and security within Databricks. • Collaborate with data engineers, modellers, and testers to ensure the platform meets technical and business requirements. • Proactively monitor, troubleshoot, and resolve issues to maintain platform stability and performance. Role Requirements • Strong platform engineering experience with Azure Databricks and related Azure services. • Familiarity with DevOps principles and CI/CD pipeline tools (e.g., Azure DevOps, Jenkins, GitHub Actions). Background in system Azure architecture, networking, Windows and Linux configuration, DevOps or SRE roles • Ability to design scalable solutions using Databricks features such as Delta Lake, Unity Catalog, and cluster policies. Write and deploy infrastructure as Code (IaC) /Configuration as Code (CaC) (Azure Resource Manager, Puppet, Ansible, Terraform or similar) • Strong problem-solving skills, with the ability to think strategically and avoid tactical or short-term solutions when possible. • Experience working in Agile/Scrum teams. Experience/Skillset • 8+ years of experience in platform engineering or a related field. • Strong knowledge of Databricks, including Spark optimization, cluster management, and governance frameworks. Good knowledge of Azure DevOps based toolchains • Proficiency in Python, SQL, and PySpark, with the ability to implement robust and efficient solutions in Databricks. • Expertise in Azure services such as Azure Data Factory, Azure Storage, and Azure Key Vault. • Strong knowledge of SDLC and Agile methodologies. • Excellent verbal and written communication skills. • Proven ability to manage multiple projects and priorities effectively while delivering to tight deadlines.
