Pyspark + Databricks

6 - 10 years

0 Lacs

Posted:1 day ago| Platform: Shine logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

As a highly skilled Technical Lead with 12 to 16 years of experience, your role will involve leading the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities. Your responsibilities will include: - Overseeing the integration of Databricks Unity Catalog to ensure seamless data governance and security across platforms. - Providing technical expertise in Databricks to optimize performance and drive innovation in data analytics. - Collaborating with cross-functional teams to deliver high-quality solutions that meet business requirements and objectives. - Developing and maintaining comprehensive documentation for data workflows and processes to facilitate knowledge sharing. - Mentoring and guiding junior team members to foster a culture of continuous learning and improvement. - Conducting regular code reviews to ensure adherence to best practices and coding standards. - Analyzing complex data sets to identify trends and insights that support strategic decision-making. - Implementing robust testing frameworks to ensure the reliability and accuracy of data solutions. - Coordinating with stakeholders to gather requirements and translate them into technical specifications. - Driving the adoption of new technologies and methodologies to enhance the team's capabilities and efficiency. - Ensuring compliance with industry standards and regulations to protect data integrity and privacy. - Monitoring system performance and troubleshooting issues to maintain optimal operation of data platforms. Qualifications: - Possessing extensive experience in Databricks Workflows and Unity Catalog demonstrating a deep understanding of their functionalities. - Having a strong background in data engineering and analytics with a proven track record of delivering successful projects. - Exhibiting excellent problem-solving skills and the ability to work effectively in a hybrid work model. - Demonstrating proficiency in programming languages such as Python or Scala relevant to Databricks environments. - Showing a keen interest in Research and Development with the ability to apply domain knowledge to data solutions. As a highly skilled Technical Lead with 12 to 16 years of experience, your role will involve leading the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities. Your responsibilities will include: - Overseeing the integration of Databricks Unity Catalog to ensure seamless data governance and security across platforms. - Providing technical expertise in Databricks to optimize performance and drive innovation in data analytics. - Collaborating with cross-functional teams to deliver high-quality solutions that meet business requirements and objectives. - Developing and maintaining comprehensive documentation for data workflows and processes to facilitate knowledge sharing. - Mentoring and guiding junior team members to foster a culture of continuous learning and improvement. - Conducting regular code reviews to ensure adherence to best practices and coding standards. - Analyzing complex data sets to identify trends and insights that support strategic decision-making. - Implementing robust testing frameworks to ensure the reliability and accuracy of data solutions. - Coordinating with stakeholders to gather requirements and translate them into technical specifications. - Driving the adoption of new technologies and methodologies to enhance the team's capabilities and efficiency. - Ensuring compliance with industry standards and regulations to protect data integrity and privacy. - Monitoring system performance and troubleshooting issues to maintain optimal operation of data platforms. Qualifications: - Possessing extensive experience in Databricks Workflows and Unity Catalog demonstrating a deep understanding of their functionalities. - Having a strong background in data engineering and analytics with a proven track record of delivering successful projects. - Exhibiting excellent problem-solving skills and the ability to work effectively in a hybrid work model. - Demonstrating proficiency in programming languages such as Python or Scala relevant to Databricks environments. - Showing a keen interest in Research and Development with the ability to apply domain knowledge to data solutions.

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Cognizant logo
Cognizant

IT Services and IT Consulting

Teaneck New Jersey

RecommendedJobs for You