Senior Databricks Architect

Title: Senior Databricks Architect Location : Hybrid ( Chennai, TN) Key Responsibilities: The role requires a strong focus on engaging with clients and showcasing thought leadership. The ideal candidate will be skilled in developing insightful content and presenting it effectively to diverse audiences Cloud Architecture & Design: • Understand customers’ overall data platform, business and IT priorities and success measures to design data solutions that drive business value. • Design scalable, high-performance, end-to-end data architectures, solutions and pipelines using Databricks Lakehouse architecture and related technologies that includes data ingestion, processing, storage and analytical capabilities. • Work with major cloud providers to integrate Databricks solutions seamlessly into customer’s enterprise environments. • Assess and validate non-functional attributes and build solutions that exhibit high levels of performance, security, scalability, maintainability, and reliability. • Optimize Databricks clusters and queries for efficiency, reliability, and cost-effectiveness Technical Leadership: • Guide technical teams in best practices on cloud adoption, migration and application modernization and provide thought leadership and insights into existing and emerging Databricks capabilities. • Ensure long term technical viability and optimization of cloud deployments by identifying and resolving bottlenecks proactively thereby ensuring high availability and efficient resource utilization. Stakeholder Collaboration: • Work closely with business leaders, developers, and operations teams to ensure alignment of technological solutions with business goals. • Work with prospective and existing customers to implement POCs/MVPs and guide through to deployment, operationalization, and troubleshooting. • Identify, communicate, and mitigate the assumptions, issues, and risks that occur throughout the project lifecycle. • Ability to judge and strike a balance between what is strategically logical and what can be accomplished realistically. Innovation and Continuous Improvement: • Stay updated with the latest advancements in Databricks and big data technologies and drive their adoption. • Implement innovative solutions to improve data processing, storage, and analytics efficiency. • Identify opportunities to enhance existing data engineering processes using AI and machine learning tools. Documentation: • Create comprehensive blueprints, architectural diagrams, technical collaterals, assets and implementation plans for Databricks solutions. Required Qualifications: Education: • Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. Experience: • Minimum of 8 years in architecture roles, including at least 3-5 years working with Databricks. Technical Expertise: • Strong proficiency in Spark, Python, Scala, SQL, and cloud platforms (Azure, AWS, GCP). • Strong proficiency in Databricks workflows, Lakehouse architecture, Delta Tables, and MLflow. • Proficiency in architectural best practices in cloud around user management, data privacy, data security, performance and other non-functional requirements. • Familiarity in building AI/ML models on cloud solutions using Databricks. Soft Skills: • Strong analytical, problem-solving and troubleshooting skills. • Excellent communication and ability to mentor and inspire teams. • Strong leadership abilities with experience managing cross-functional teams. Preferred Skills: • Databricks certification as a professional or architect. • Experience with BFSI or Healthcare or Retail domain. • Experience with hybrid cloud environments and multi-cloud strategies. • Experience with data governance principles, data privacy and security. • Experience with data visualization tools like Power BI or Tableau. Show more Show less