SAP PIPO - Kankana (MSQUBE)

As a Databricks Administrator, your role involves managing the provisioning, configuration, and administration of Databricks workspaces. This includes tasks such as setting up and monitoring clusters, jobs, pools, and libraries to ensure optimal performance and cost-efficiency. You will also be responsible for administering user roles, groups, and access controls using SCIM, Unity Catalog, and Identity Federation for seamless Single Sign-On (SSO) integration. Your key responsibilities include: - Implementing security best practices to control data access, enable credential passthrough, and maintain audit logging. - Managing CI/CD pipelines for Databricks notebooks, workflows, and infrastructure using tools like Git, Azure DevOps, or Jenkins. - Performing capacity planning, usage monitoring, and cost optimization to enhance the Databricks environment's efficiency. - Supporting data engineers, analysts, and ML engineers in cluster setup, job scheduling, and debugging. - Monitoring job and cluster performance, providing proactive troubleshooting, and ensuring timely resolution of issues. - Handling Databricks REST APIs and automating tasks using tools like Terraform, Databricks CLI, or Workspace APIs. - Ensuring compliance with enterprise security, data governance, and regulatory standards such as GDPR and HIPAA. - Collaborating with cloud teams to integrate Databricks with storage, data catalogs, and logging platforms like AWS S3, Azure ADLS, Unity Catalog, CloudWatch, and Azure Monitor. - Regularly updating, patching, and upgrading platform components in accordance with vendor best practices.,