Description
Staples is business to business. Youre what binds us together.Youre what binds us together. Our digital solutions team is more than a traditional IT organization. We are a team of passionate, collaborative, agile, inventive, customer-centric, results-oriented problem solvers. We are intellectually curious, love advancements in technology, and seek to adapt technologies to drive Staples forward. We anticipate the needs of our customers and business partners and deliver reliable, customer-centric technology services.The Databricks Platform Architect will be responsible for the strategic design, implementation, and governance of our Databricks platform. This role defines best practices, establishes architectural patterns, and provides expert guidance to data engineers, data scientists, and other stakeholders to ensure the scalability, security, performance, and cost-efficiency of our data and machine learning initiatives. The ideal candidate will possess deep expertise in Databricks, Apache Spark, and cloud data architectures across one or more major cloud providers (AWS, Azure, GCP), with a strong understanding of AI/ML Ops principles and practices.
What You'll Be Doing
- Lead the architectural design and evolution of the Databricks platform, establishing best practices for security, governance, cost optimization, and performance.
- Design and implement robust ML/AI Ops frameworks and pipelines on Databricks, supporting the full machine learning lifecycle.
- Provide architectural recommendations and technical guidance to stakeholders and internal teams.
- Oversee the configuration and management of Databricks workspaces, clusters, and resources.
- Implement and manage data orchestration workloads and integrate Databricks with various data sources and BI tools.
- Implement and manage tools for ML experiment tracking (MLFlow), model registry, versioning, and deployment on Databricks.
- Drive the adoption of Infrastructure as Code (IaC) principles and develop CI/CD pipelines for Databricks assets, including ML models.
- Optimize Databricks environments for storage and compute performance, actively monitoring platform health and managing costs.
- Implement enterprise-grade governance and security measures, leveraging Unity Catalog for unified data and ML asset governance.
- Collaborate closely with data engineers, data scientists, DevOps teams, and business stakeholders to deliver robust solutions.
- Act as a trusted advisor, guiding technical decisions and fostering skills development within the team.
- Stay current with technological trends in the data and AI space, proactively identifying innovative solutions.
- Assist in architecting, developing, deploying, and migrating use cases and workloads into production on Databricks Platform.
- Operate as an expert solution architect and trusted advisor for the Databricks Data Intelligence Platform, developing multi-language notebooks for Data Engineering and Data Science (AI/ML).
Required Skills And Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
- 5+ years of experience in data architecture, data engineering, or a related field, with significant experience specifically on the Databricks platform.
- Deep expertise in Databricks architecture, including clusters, notebooks, jobs, Delta Lake, and Unity Catalog.
- Hands-on experience with Databricks implementation on Azure.
- Proficiency in Apache Spark, including Spark SQL, Spark Streaming, and MLlib.
- Strong programming skills in Python and SQL.
- Extensive hands-on experience with at least one major cloud platform (Azure, AWS, or Google Cloud Platform), including its core services for storage (ADLS, S3, GCS), networking (VPCs, VNETs), identity and access management, and data security.
- Proven experience in designing and implementing data lakehouses, ETL/ELT workflows, AI/ML workflows and data modeling.
- Demonstrated experience with Machine Learning Operations (MLOps) principles and tools, including model versioning, experiment tracking (e.g., MLFlow), model deployment strategies (e.g., REST APIs, batch inference), and model monitoring.
- Experience with CI/CD practices and Infrastructure as Code (e.g., Terraform, Azure DevOps, Jenkins).
- Solid understanding of data security principles. Expertise in implementing Row Level Security, Column Masking, Data encryption, and decryption solutions in Databricks.
- Excellent communication, interpersonal, and leadership skills with the ability to articulate complex technical concepts to diverse audiences.
- Proven experience with Snowflake, DBT.
(ref:hirist.tech)
