Databrciks Administrator

Primary Skills

• Databricks Engineer:

  7+ yrs of total experience with around 3 yrs in leading and overseeing the design, development, and management of the data infrastructure on the Databricks platform within an AWS / Azure cloud environment.
• Create new Databricks workspaces (premium, standard, serverless) and clusters including right sizing
• Drop unused workspaces
• Delta Sharing: Work with enterprise teams on connected data (data sharing)
• User Management \ Create new security groups and add/delete users
• Assign Unity Catalog permissions to respective groups/teams
• Review and analyze Databricks logs and error messages.
• Identify and address problems related to cluster configuration or job failures.
• Optimize Databricks notebooks and jobs for performance.
• Develop and test Databricks clusters to ensure stability and scalability.
• Outlines the security and compliance obligations of Databricks. Creating and maintaining database standards and policies. Administering database objects to achieve optimum utilization.
• Mentor team members on cluster management, job optimization, and resource allocation within Databricks environments
• Ensure adherence to compliance standards and maintain platform security
• Drive adoption of advanced capabilities in Databricks like Photon and Graviton instances for improved efficiency
• Regularly update and refine existing architectures to meet changing business and technology needs

• Cloud computing expertise

  : A strong understanding of cloud computing services, including cloud infrastructure, software, and platform as a service (IaaS, SaaS, and PaaS). Proficiency in cloud platforms (AWS, Azure), networking, security, programming, scripting, database management, automation tools.

Secondary / or Good to Have Skills.

• Experience on DevOps:

  Preferably Terraform and GIT to develop WF for automation.