Generative AI Architect

We are seeking a highly skilled and experienced AWS Architect with a strong background in Data Engineering and expertise in Generative AI. In this pivotal role, you will be responsible for designing, building, and optimizing scalable, secure, and cost-effective data solutions that leverage the power of AWS services, with a particular focus on integrating and managing Generative AI capabilities. The ideal candidate will possess a deep understanding of data architecture principles, big data technologies, and the latest advancements in Generative AI, including Large Language Models (LLMs) and Retrieval Augmented Generation (RAG). You will work closely with data scientists, machine learning engineers, and business stakeholders to translate complex requirements into robust and innovative solutions on the AWS platform.

Responsibilities:

• Architect and Design: Lead the design and architecture of end-to-end data platforms and pipelines on AWS, incorporating best practices for scalability, reliability, security, and cost optimization.

• Generative AI Integration: Architect and implement Generative AI solutions using AWS services like Amazon Bedrock, Amazon SageMaker, Amazon Q, and other relevant technologies. This includes designing RAG architectures, prompt engineering strategies, and fine-tuning models with proprietary data (knowledge base).

• Data Engineering Expertise: Design, build, and optimize ETL/ELT processes for large-scale data ingestion, transformation, and storage using AWS services such as AWS Glue, Amazon S3, Amazon Redshift, Amazon Athena, Amazon EKS and Amazon EMR.

• Data Analytics: Design, build, and optimize analytical solutions for large-scale data ingestion, analytics and insights using AWS services such as AWS Quicksight

• Data Governance and Security: Implement robust data governance, data quality, and security measures, ensuring compliance with relevant regulations and industry best practices for both traditional data and Generative AI applications.

• Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and Generative AI workloads, ensuring efficient resource utilization and optimal response times.

• Technical Leadership: Act as a subject matter expert and provide technical guidance to data engineers, data scientists, and other team members. Mentor and educate on AWS data and Generative AI best practices.

• Collaboration: Work closely with cross-functional teams, including product owners, data scientists, and business analysts, to understand requirements and deliver impactful solutions.

• Innovation and Research: Stay up-to-date with the latest AWS services, data engineering trends, and advancements in Generative AI, evaluating and recommending new technologies to enhance our capabilities.

• Documentation: Create comprehensive technical documentation, including architectural diagrams, design specifications, and operational procedures.

• Cost Management: Monitor and optimize AWS infrastructure costs related to data and Generative AI workloads.

Required Skills and Qualifications:

• 12+ years of experience in data engineering, data warehousing, or big data architecture.

• 5+ years of experience in an AWS Architect role, specifically with a focus on data.

• Proven experience designing and implementing scalable data solutions on AWS.

• Strong hands-on experience with core AWS data services, including:

o Data Storage: Amazon S3, Amazon Redshift, Amazon DynamoDB, Amazon RDS

o Data Processing: AWS Glue, Amazon EMR, Amazon EKS, AWS Lambda, Informatica

o Data Analytic: Amazon Quicksight, Amazon Athena, Tableau

o Data Streaming: Amazon Kinesis, AWS MSK

o Data Lake: AWS Lake Formation

• Strong competencies in Generative AI, including:

o Experience with Large Language Models (LLMs) and Foundation Models (FMs).

o Hands-on experience with Amazon Bedrock (including model customization, agents, and orchestrations).

o Understanding and experience with Retrieval Augmented Generation (RAG) architectures and vector databases (e.g., Amazon OpenSearch Service for vector indexing).

o Experience with prompt engineering and optimizing model responses.

o Familiarity with Amazon SageMaker for building, training, and deploying custom ML/Generative AI models.

o Knowledge of Amazon Q for business-specific Generative AI applications.

• Proficiency in programming languages such as Python (essential), SQL, and potentially Scala or Java.

• Experience with MLOps/GenAIOps principles and tools for deploying and managing Generative AI models in production.

• Solid understanding of data modeling, data warehousing concepts, and data lake architectures.

• Experience with CI/CD pipelines and DevOps practices on AWS.

• Excellent communication, interpersonal, and presentation skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.

• Strong problem-solving and analytical abilities.

Preferred Qualifications:

• AWS Certified Solutions Architect – Professional or AWS Certified Data Engineer – Associate/Specialty.

• Experience with other Generative AI frameworks (e.g., LangChain) or open-source LLMs.

• Familiarity with containerization technologies like Docker and Kubernetes (Amazon EKS).

• Experience with data transformation tools like Informatica, Matillion

• Experience with data visualization tools (e.g., Amazon QuickSight, Tableau, Power BI).

• Knowledge of data governance tools like Amazon DataZone.

• Experience in a highly regulated industry (e.g., Financial Services, Healthcare).