Job
Description
Core Responsibilities:- Platform Management : Oversee the deployment, configuration, and maintenance of Databricks clusters and workspaces. Security & Access Control : Implement role-based access controls (RBAC), manage Unity Catalog, and configure service principals and access tokens. Performance Monitoring : Monitor cluster health, optimize resource utilization, and troubleshoot performance issues. Automation & Scripting : Automate administrative tasks using tools like Python, PowerShell, and Terraform. Integration & Deployment : Manage integrations with Azure Data Lake, Key Vault, and implement CI/CD pipelines using Azure DevOps. Compliance & Governance : Ensure data governance, implement backup and disaster recovery strategies, and adhere to security best practices. User Support & Training : Provide technical support, conduct training sessions, and maintain documentation. Required Skills & Experience Cloud Platforms : Proficiency in Azure, AWS, or GCP; Azure experience is often preferred. Programming Languages : Strong skills in Python, PySpark, PowerShell, and SQL. Infrastructure as Code : Experience with Terraform for provisioning and managing cloud resources. Data Engineering : Knowledge of ETL processes, data pipelines, and big data technologies. Security & Compliance : Understanding of data security principles, compliance requirements, and best practices. Experience : Typically, 5-10 years in IT administration, with at least 25 years focused on Databricks administration.
