Lead Databricks Analyst

Location:

Employment Type

Overview

Rialtes is seeking a Databricks Support Analyst to manage platform operations, ensure reliability of clusters and jobs, and support development of data workflows. Role requires hands-on experience with Databricks administration, SQL-based development, and basic Python.

Key Responsibilities

Primary: Platform Support

• Monitor and maintain Databricks clusters, jobs, workflows, and schedules.
• Troubleshoot issues related to compute resources, job failures, data ingestion completeness, schema changes, and data accuracy.
• Manage user provisioning, access controls, workspace permissions, and catalogs using Unity Catalog aligned with governance standards.
• Ensure cost-optimized cluster usage and enforce security, tagging, and compliance policies.
• Collaborate with Cloud Ops, Data Engineering, and InfoSec teams to resolve platform-wide incidents.

Secondary: Design & Development

• Assist in building Databricks workflows for ETL, analytics, and machine learning pipelines.
• Support development and optimization of notebooks and scripts—primarily SQL, with some Python for transformations and validations.
• Implement best practices for job cluster configuration, Photon acceleration, Delta Lake, and Unity Catalog integration.
• Participate in unit testing, functional testing, UAT cycles, and production migration of Databricks jobs.
• Contribute to documentation including runbooks, SOPs, job catalogs, and data lineage.

Qualifications

• 5 years of experience in Databricks, data engineering support, or cloud platform operations.
• Strong SQL skills; working knowledge of Python preferred.
• Hands-on experience with Databricks clusters, job orchestration, Delta Lake, and Unity Catalog.
• Understanding of ETL workflows, CI/CD for data pipelines, and cloud environments (AWS/Azure/GCP).
• Strong debugging skills for job failures, performance bottlenecks, and ingestion issues.
• Ability to work in fast-paced environments with cross-functional teams.

Nice-to-Have

• Databricks Certified Associate or equivalent.
• Exposure to MLflow, Data Quality (DQ) frameworks, or Databricks Workflows orchestration.
• Experience with Git, DevOps tools, or monitoring solutions (CloudWatch, Azure Monitor, etc.).