Job
Description
About Position: We are seeking a highly experienced and results-driven Databricks Competency Leader with over 20 years of experience in data engineering, analytics, and cloud technologies. The ideal candidate will have a strong command of Databricks technologies and will lead the development and execution of advanced data solutions on the Databricks platform. You will be responsible for establishing the organization’s Databricks competencies, ensuring best practices, optimizing workflows, and delivering high-performance data pipelines. This is a key leadership role, responsible for both technical and strategic leadership in driving the adoption of Databricks across the practice. Role: Chief Architect - Competency Head Location: Pune Experience: 20-25 Years Job Type: Full time What You'll Do: Leadership and Strategy : Lead the Databricks competency center, establishing the vision, strategy, and roadmap for Databricks adoption, implementation, and optimization across the organization. Team Management & Mentoring : Build, mentor, and manage a cross-functional team of data engineers, data scientists, and architects with expertise in Databricks, Spark, and cloud technologies. Foster a culture of collaboration, innovation, and continuous learning. Databricks Platform Expertise : Serve as the subject matter expert (SME) for Databricks, ensuring the platform is leveraged effectively to meet business requirements. Drive the adoption of Databricks for large-scale data processing, machine learning, and analytics. Solution Architecture & Design: Oversee the design, development, and optimization of Databricks-based data pipelines, frameworks, and architectures. Ensure the scalability, reliability, and performance of data solutions. Best Practices & Standards: Establish best practices for Databricks implementations, including coding standards, pipeline optimization, performance tuning, data governance, and security compliance. Stakeholder Collaboration : Work closely with senior leadership, data teams, and business units to align Databricks initiatives with organizational goals. Ensure that Databricks solutions are integrated with other data platforms and business applications effectively. Innovation & Continuous Improvement: Stay on top of industry trends and advancements in cloud technologies, particularly Databricks, Apache Spark, and related tools, and implement new features and capabilities to drive business value. Project Delivery & Execution: Lead the execution of Databricks projects, ensuring timely delivery, quality, and meeting performance and scalability requirements. Oversee project scoping, resourcing, and budget management. Training & Enablement: Provide training, workshops, and resources to upskill internal teams on Databricks tools and technologies, ensuring self-sufficiency and competency in using the platform. Expertise You'll Bring: Bachelor’s or Master’s degree in computer science, Engineering, Data Science, or a related field. Advanced certifications in cloud platforms (AWS, Azure, Google Cloud) and Databricks are preferred. Over 20 years of experience in data engineering, data architecture, and cloud technologies, with at least 10 years specifically focused on Databricks or Apache Spark. Proven experience in leading large-scale data initiatives using Databricks, including the design and implementation of complex data pipelines, analytics, and machine learning workflows. Extensive experience in leading and building high-performing teams of data engineers, scientists, and architects. Strong understanding of cloud platforms (AWS, Azure, GCP) and integration of Databricks with other cloud-native tools and services. Expertise in Databricks, Apache Spark, Delta Lake, and related big data technologies. Proficiency in Python, Scala, SQL, and other relevant programming languages for data processing and analytics. Hands-on experience with MLflow, Databricks SQL, and other advanced Databricks features. Strong experience with cloud-native data architectures, real-time data streaming, and data lake solutions. Knowledge of data governance, security, and compliance standards (e.g., GDPR, HIPAA). Strong leadership and management skills, with the ability to motivate teams and drive collaboration across functional areas. Exceptional communication skills, with the ability to interact with senior stakeholders and present complex technical concepts to non-technical audiences. Strategic thinking, with a proven ability to balance technical excellence with business objectives. Experience in multiple Databricks environments, including migrations, platform upgrades, and multi-cloud solutions. Prior experience in consulting or working with enterprise clients on large-scale data and analytics transformations. Knowledge of machine learning, AI, and data science applications using Databricks and related tools. Certification in Databricks, Apache Spark, or relevant cloud platforms (AWS, Azure, GCP) Benefits: Competitive salary and benefits package Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications. Opportunity to work with cutting-edge technologies. Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards Annual health check-ups Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents Inclusive Environment: Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds. We offer hybrid work options and flexible working hours to accommodate various needs and preferences. Our office is equipped with accessible facilities, including adjustable workstations, ergonomic chairs, and assistive technologies to support employees with physical disabilities. If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment. We are committed to creating an inclusive environment where all employees can thrive. Our company fosters a value-driven and people-centric work environment that enables our employees to : Accelerate growth, both professionally and personally Impact the world in powerful, positive ways, using the latest technologies. Enjoy collaborative innovation, with diversity and work-life wellbeing at the core. Unlock global opportunities to work and learn with the industry’s best. Let’s unleash your full potential at Persistent. “Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”
