Data Platform Architect

Overview

Data Platform Architect

Responsibilities

Key ResponsibilitiesArchitectural Vision & Strategy: Design and own the end-to-end architecture for our data platform,encompassing data ingestion, processing, storage, and consumption layers.System Design: Create detailed architectural diagrams, design documents, and technical specificationsfor new platform capabilities. Define the standards for how services interact.Technology Evaluation & Governance: Research, evaluate, and prototype new technologies (e.g.,table formats, query engines, streaming platforms). Establish and govern technical standards and bestpractices across all data engineering teams.Cross-Functional Leadership: Collaborate with engineering leaders, product managers, data scientists,and business stakeholders to ensure the platform architecture meets current and future needs. Act asthe key technical advisor for all things data.Non-Functional Requirements: Drive the strategy for key architectural concerns, including scalability,security, data governance, performance, and cost optimization.Prototyping & Frameworks: Develop proof-of-concepts and foundational frameworks thatengineering teams can build upon, ensuring architectural principles are correctly implemented

Requirements

Required Qualifications10+ years of software engineering experience, with at least 4 years in a dedicated Architect role.Proven experience designing and architecting large-scale, distributed data systems from theground up.Deep expertise in the modern data stack, including deep knowledge of Apache Spark, Kubernetes,and object storage systems like MinIO or S3.Extensive experience with open table formats like Apache Iceberg, Delta Lake, or Hudi, and astrong understanding of their internal workings.Demonstrated ability to think strategically, influence without authority, and communicate complextechnical concepts to both technical and non-technical audiences.Experience creating formal architectural artifacts (e.g., ADRs, design docs, sequence diagrams).

Preferred Qualifications

Experience designing multi-tenant data platforms.Expertise with interactive MPP query engines like Trino or Presto.Knowledge of data governance and data cataloging solutions.Experience with both batch and stream processing architectures (e.g., Kafka, Flink).Familiarity with Infrastructure as Code (Terraform, Pulumi).