Description
We are looking for a Data Engineer to join our Agentic Data Platform team and help build the next generation of Amazon's Partner analytics infrastructure. You will work at the intersection of large-scale data engineering and agentic AI — designing systems that power 4,000+ standardized metrics daily, serve 900K+ Selling Partners and 30K+ Amazonians, and enable AI-driven automation across the Parter ecosystem of tools and products.
Key job responsibilities
-
You will build the foundational data layer that agentic systems depend on — schema-aware, version-controlled, and optimized for programmatic access.
-
You will engineer event-driven pipelines (Spark Structured Streaming, Kafka, Kinesis) that serve as the nervous system for agentic workflows — providing the low-latency data feeds that agents consume to trigger actions, detect anomalies, and orchestrate downstream processes.
-
You will define how agents discover data, resolve ambiguity across metric definitions, and compose multi-step analytical queries against our Curated Data Layer (CDL).
-
You will build intelligent agents that own the full DataOps loop: auto-detecting schema drift, self-healing broken pipelines, orchestrating metric onboarding workflows, running data quality checks, and publishing validated datasets — all with minimal human intervention.
-
You will design the trust layer for agentic data systems: output validation agents that verify SQL correctness before execution, confidence scoring for AI-generated metric definitions, automated regression testing against known-good baselines, and escalation workflows that route low-confidence outputs to human reviewers.
-
You will lead the transformation from static, manually-maintained pipelines to agent-operated infrastructure — migrating legacy ETL to Iceberg-based incremental processing, unifying metric definitions across 12+ surfaces through agent-driven reconciliation, and building the operational backbone that enables 4,000+ metrics to be governed, published, and consumed by both humans and AI agents.
Basic Qualifications
-
3+ years of data engineering experience
-
4+ years of SQL experience
-
Experience with data modeling, warehousing and building ETL pipelines
-
Experience building large-scale, high-throughput, 24x7 data systems
-
Experience coding, scripting, automating in at least one language (e.g., Python, C, C++, Java, Ruby, or PowerShell)
Preferred Qualifications
-
Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
-
Familiarity with API design patterns (REST, GraphQL, or MCP) for exposing data services to downstream applications, including GenAI agents and LLM-powered systems
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
null
Company
- ADCI - Tamil Nadu
Job ID: A10602102