Module Lead Databricks Administrator

Required Skills & Experience:
6+ years of experience in Data Platform Administration, with 2+ years specifically on Databricks. Strong hands-on experience with Databricks on Azure. Strong knowledge of Spark, Delta Lake, and notebook lifecycle management. Proficient in scripting languages (Python, Bash, PowerShell). Experience with CI/CD tools (e.g., Azure DevOps, GitHub Actions, Jenkins). Familiarity with data governance, RBAC, and security practices in Databricks. Excellent problem-solving, communication, and leadership skills.

Preferred Qualifications:
Databricks Certified Associate/Professional Administrator or Data Engineer. Experience with infrastructure-as-code (Terraform, ARM templates). Exposure to MLOps and integration with MLflow. Experience managing multi-tenant or enterprise-scale Databricks environments.

Platform Administration:
Install, configure, and maintain Databricks workspaces. Manage user access, cluster policies, pools, libraries, and workspace configurations. Monitor performance, perform tuning, and resolve operational issues. Oversee job scheduling, automation, and alerts.

Team Leadership:
Lead a team of junior administrators and engineers in delivering high-performance Databricks solutions. Assign and track tasks, ensuring project milestones and SLA targets are met. Mentor team members and promote best practices in Databricks administration and data governance.

Operational Support:
Respond to incidents and service requests related to Databricks. Coordinate with cloud platform teams (Azure) for networking, security, and integration requirements. Develop documentation for platform usage, incident handling, and standard operating procedures.

Optimization & Innovation:
Analyze usage patterns and recommend cost optimization strategies. Research and implement new Databricks features and updates. Propose automation and CI/CD practices for efficient platform management.