Data Engineer

Data Engineer - Senior Consultant - Data Engineer

Exp

Package

Notice Period

Location

Work Mode

Interview mode

Mandatory Skills:

Responsibilities

• Design, build, and optimize platform solutions in Azure Databricks to support scalable, high-performance, and secure data engineering workflows.

• Implement and maintain platform automation for infrastructure, monitoring, and deployment pipelines using modern DevOps practices.

• Align with best practices for platform architecture, governance, and security within Databricks.

• Collaborate with data engineers, modelers, and testers to ensure the platform meets technical and business requirements.

• Proactively monitor, troubleshoot, and resolve issues to maintain platform stability and performance.

Role Requirements

• Strong platform engineering experience with Azure Databricks and related Azure services.

• Familiarity with DevOps principles and CI/CD pipeline tools (e.g., Azure DevOps, Jenkins, GitHub Actions).

• Background in system Azure architecture, networking, Windows and Linux configuration, DevOps or SRE roles

• Ability to design scalable solutions using Databricks features such as Delta Lake, Unity Catalog, and cluster policies.

• Write and deploy infrastructure as Code (IaC) /Configuration as Code (CaC) (Azure Resource Manager, Puppet, Ansible, Terraform or similar)

• Strong problem-solving skills, with the ability to think strategically and avoid tactical or short-term solutions when possible.

• Experience working in Agile/Scrum teams.

Experience/Skillset

• 8+ years of experience in platform engineering or a related field.

• Strong knowledge of Databricks, including Spark optimization, cluster management, and governance frameworks.

• Good knowledge of Azure DevOps-based tool chains

• Proficiency in Python, SQL, and PySpark, with the ability to implement robust and efficient solutions in Databricks.

• Expertise in Azure services such as Azure Data Factory, Azure Storage, and Azure Key Vault

• Strong knowledge of Data modeling.

• Strong knowledge of SDLC and Agile methodologies.

• Excellent verbal and written communication skills.

• Proven ability to manage multiple projects and priorities effectively while delivering to tight deadlines.

Time Passer Please excuse. Only JD matching candidates please share me your profiles to victorpauls@gmail.com

Connect with me in LinkedIn for Swift response. Only screen shortlisted profiles will be contacted further.

With Regards,

Victor

#Data #DataEngineer #AzureDatabricks #Python #SQL #Pyspark #DataModeling