Data Platform Architect

Prodapt Solutions

10 - 15 years

17 - 20 Lacs

chennai

Posted:14 hours ago| Platform:

Apply

Skills Required

software engineering s3 kubernetes prototyping data cataloging data engineering flink spark stream processing apache spark kafka data governance storage systems terraform infrastructure as code

Work Mode

Work from Office

Job Type

Full Time

Job Description

Overview

Key Responsibilities

Architectural Vision & Strategy: Design and own the end-to-end architecture for our data platform, encompassing data ingestion, processing, storage, and consumption layers
System Design: Create detailed architectural diagrams, design documents, and technical specifications for new platform capabilities
Define the standards for how services interact
Technology Evaluation & Governance: Research, evaluate, and prototype new technologies (e. g,table formats, query engines, streaming platforms)
Establish and govern technical standards and bestpractices across all data engineering teams
Cross-Functional Leadership: Collaborate with engineering leaders, product managers, data scientists,and business stakeholders to ensure the platform architecture meets current and future needs
Act asthe key technical advisor for all things data
Non-Functional Requirements: Drive the strategy for key architectural concerns, including scalability,security, data governance, performance, and cost optimization
Prototyping & Frameworks: Develop proof-of-concepts and foundational frameworks thatengineering teams can build upon, ensuring architectural principles are correctly implemented

Requirements

Required Qualifications

10+ years of software engineering experience, with at least 4 years in a dedicated Architect role
Proven experience designing and architecting large-scale, distributed data systems from the ground up
Deep expertise in the modern data stack, including deep knowledge of Apache Spark, Kubernetes,and object storage systems like MinIO or S3
Extensive experience with open table formats like Apache Iceberg, Delta Lake, or Hudi, and astrong understanding of their internal workings
Demonstrated ability to think strategically, influence without authority, and communicate complextechnical concepts to both technical and non-technical audiences
Experience creating formal architectural artifacts (e. g, ADRs, design docs, sequence diagrams)

Preferred Qualifications

Experience designing multi-tenant data platforms
Expertise with interactive MPP query engines like Trino or Presto
Knowledge of data governance and data cataloging solutions
Experience with both batch and stream processing architectures (e.g, Kafka, Flink)
Familiarity with Infrastructure as Code (Terraform, Pulumi).

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.