AWS+Gen/Agentic AI Lead

Lead Data Engineer - Generative AI & Agentic Systems (AWS)

About the Role

We are seeking a Lead Data Engineer with strong expertise in AWS-based data engineering and hands-on production experience with Generative AI and Agentic AI systems. This role blends deep data infrastructure engineering with modern GenAI application development, focusing on scalable, secure, and production-grade AI systems.

As a Lead, you will own solution architecture, guide engineering best practices, and mentor team members while remaining deeply hands-on. You will play a critical role in taking GenAI initiatives from experimentation to enterprise-scale deployment.

Key Responsibilities

Technical Leadership & Architecture

• Lead the design and implementation of scalable data and GenAI architectures on AWS.


• Define best practices, coding standards, and architectural patterns for GenAI and data engineering teams.


• Review designs and code to ensure performance, security, reliability, and cost optimization.


• Act as a technical mentor and escalation point for complex engineering challenges.

Data Engineering & Cloud Infrastructure

• Design, build, and maintain robust data pipelines and AI data infrastructure using AWS services such as Glue, Lambda, S3, Redshift, Athena, Step Functions, and related services.


• Develop high-performance PySpark pipelines for large-scale data processing in production environments.


• Ensure data quality, lineage, governance, and reliability across platforms.

Generative AI & Agentic AI Systems

• Design and deploy LLM-powered applications using frameworks such as LangChain, LlamaIndex, AutoGen, or equivalent.


• Build and maintain Retrieval-Augmented Generation (RAG) pipelines, integrating S3, Bedrock, SageMaker, and vector databases (OpenSearch, Pinecone, FAISS, Chroma, Milvus, etc.).


• Implement agentic reasoning, tool invocation, memory, and orchestration for single-agent and multi-agent workflows.


• Integrate LLMs with internal data systems, APIs, and enterprise applications.

Deployment, Observability & Operations

• Containerize and deploy AI services using Docker, ECS, or EKS, ensuring scalability and reliability.


• Design and maintain CI/CD pipelines for data and AI workloads.


• Implement monitoring, logging, tracing, and model observability using CloudWatch, X-Ray, and LLMOps tools.


• Optimize runtime performance, latency, and infrastructure cost for AI workloads.

Collaboration & Delivery

• Collaborate closely with ML engineers, data scientists, product teams, and cloud architects to deliver production-ready GenAI solutions.


• Drive the transition from PoCs to enterprise-grade systems, ensuring compliance, security, and operational readiness.

Required Skills & Experience

• 7-11 years of experience in Data Engineering, with strong hands-on AWS expertise.


• Advanced proficiency in Python and PySpark, with real-world production experience.


• Proven experience delivering GenAI solutions in production environments (beyond demos or PoCs).


• Hands-on experience with Agentic AI frameworks (LangChain, LlamaIndex, AutoGen, or similar).


• Strong understanding of RAG architectures, vector databases, and embedding strategies.


• Experience with LLMOps, prompt lifecycle management, evaluation, and performance monitoring.


• Practical experience deploying workloads on AWS ECS/EKS, building CI/CD pipelines, and managing production systems.


• Solid knowledge of AWS security best practices, including IAM, VPC, Secrets Manager, and data protection.


• Strong problem-solving, communication, and technical leadership skills.

Lead Data Engineer - Generative AI & Agentic Systems (AWS)

About the Role

We are seeking a Lead Data Engineer with strong expertise in AWS-based data engineering and hands-on production experience with Generative AI and Agentic AI systems. This role blends deep data infrastructure engineering with modern GenAI application development, focusing on scalable, secure, and production-grade AI systems.

As a Lead, you will own solution architecture, guide engineering best practices, and mentor team members while remaining deeply hands-on. You will play a critical role in taking GenAI initiatives from experimentation to enterprise-scale deployment.

Key Responsibilities

Technical Leadership & Architecture

• Lead the design and implementation of scalable data and GenAI architectures on AWS.


• Define best practices, coding standards, and architectural patterns for GenAI and data engineering teams.


• Review designs and code to ensure performance, security, reliability, and cost optimization.


• Act as a technical mentor and escalation point for complex engineering challenges.

Data Engineering & Cloud Infrastructure

• Design, build, and maintain robust data pipelines and AI data infrastructure using AWS services such as Glue, Lambda, S3, Redshift, Athena, Step Functions, and related services.


• Develop high-performance PySpark pipelines for large-scale data processing in production environments.


• Ensure data quality, lineage, governance, and reliability across platforms.

Generative AI & Agentic AI Systems

• Design and deploy LLM-powered applications using frameworks such as LangChain, LlamaIndex, AutoGen, or equivalent.


• Build and maintain Retrieval-Augmented Generation (RAG) pipelines, integrating S3, Bedrock, SageMaker, and vector databases (OpenSearch, Pinecone, FAISS, Chroma, Milvus, etc.).


• Implement agentic reasoning, tool invocation, memory, and orchestration for single-agent and multi-agent workflows.


• Integrate LLMs with internal data systems, APIs, and enterprise applications.

Deployment, Observability & Operations

• Containerize and deploy AI services using Docker, ECS, or EKS, ensuring scalability and reliability.


• Design and maintain CI/CD pipelines for data and AI workloads.


• Implement monitoring, logging, tracing, and model observability using CloudWatch, X-Ray, and LLMOps tools.


• Optimize runtime performance, latency, and infrastructure cost for AI workloads.

Collaboration & Delivery

• Collaborate closely with ML engineers, data scientists, product teams, and cloud architects to deliver production-ready GenAI solutions.


• Drive the transition from PoCs to enterprise-grade systems, ensuring compliance, security, and operational readiness.

Required Skills & Experience

• 7-11 years of experience in Data Engineering, with strong hands-on AWS expertise.


• Advanced proficiency in Python and PySpark, with real-world production experience.


• Proven experience delivering GenAI solutions in production environments (beyond demos or PoCs).


• Hands-on experience with Agentic AI frameworks (LangChain, LlamaIndex, AutoGen, or similar).


• Strong understanding of RAG architectures, vector databases, and embedding strategies.


• Experience with LLMOps, prompt lifecycle management, evaluation, and performance monitoring.


• Practical experience deploying workloads on AWS ECS/EKS, building CI/CD pipelines, and managing production systems.


• Solid knowledge of AWS security best practices, including IAM, VPC, Secrets Manager, and data protection.


• Strong problem-solving, communication, and technical leadership skills.