AWS Big Data with AI

Role & responsibilities

• Data Architecture & Pipelines:

  Design, build, and optimize scalable ETL/ELT data pipelines, data lakes (Amazon S3), and data warehouses (Amazon Redshift, Snowflake, or Athena). [1, 2, 3]

• AI & GenAI Integration:

  Implement AI/ML and Generative AI solutions using

  Amazon Bedrock

  ,

  Amazon SageMaker AI

  , vector databases, and Retrieval-Augmented Generation (RAG). [1, 2]

• Automation & Analytics:

  Automate reporting, enable self-service analytics at scale, and utilize AI agents or natural language querying tools (such as Amazon QuickSight or custom LLM agents) to extract business insights. [1, 2, 3]

• Data Governance & Quality:

  Ensure data quality, reliability, security, IAM access controls, and compliance through monitoring, validation, and Infrastructure as Code (IaC). [1, 2]

• Collaboration:

  Partner with product managers, data scientists, and business stakeholders to translate business problems into technical data and AI requirements. [1, 2]

Required Skills & Technical Stack

• Cloud & Big Data (AWS):

• Storage & Compute: Amazon S3, AWS Lambda, Amazon EMR, Amazon Kinesis.
• Analytics & Warehousing: Amazon Athena, Amazon Redshift, AWS Glue, Apache Spark. [1, 2, 3, 4]

• AI & Machine Learning:

• Frameworks & Services: Amazon Bedrock, Amazon SageMaker AI, LLMs, embeddings, vector search, and RAG implementation patterns. [1]

• Programming & Scripting:

• Strong proficiency in

  Python

  and

  SQL

  .
• Experience with transformation and orchestration tools like

  dbt

  ,

  Apache Airflow

  , or

  Dagster

  . [1, 2, 3]

• Infrastructure & DevOps:

• Infrastructure as Code (IaC) using

  Terraform

  ,

  AWS CDK

  , or

  CloudFormation

  , paired with CI/CD pipelines. [1]

Qualifications & Experience

• Education:

  Bachelors or Master’s degree in Computer Science, Engineering, Information Technology, or a related quantitative field. [1, 2]

• Experience:

  3+ to 5+ years of hands-on experience in data engineering, cloud architecture, or analytics, with a growing portfolio of applied AI/ML or Generative AI projects. [1, 2]

Preferred candidate profile

• Data Architecture & Pipelines:

  Design, build, and optimize scalable ETL/ELT data pipelines, data lakes (Amazon S3), and data warehouses (Amazon Redshift, Snowflake, or Athena). [1, 2, 3]

• AI & GenAI Integration:

  Implement AI/ML and Generative AI solutions using

  Amazon Bedrock

  ,

  Amazon SageMaker AI

  , vector databases, and Retrieval-Augmented Generation (RAG). [1, 2]

• Automation & Analytics:

  Automate reporting, enable self-service analytics at scale, and utilize AI agents or natural language querying tools (such as Amazon QuickSight or custom LLM agents) to extract business insights. [1, 2, 3]

• Data Governance & Quality:

  Ensure data quality, reliability, security, IAM access controls, and compliance through monitoring, validation, and Infrastructure as Code (IaC). [1, 2]

• Collaboration:

  Partner with product managers, data scientists, and business stakeholders to translate business problems into technical data and AI requirements. [1, 2]

Required Skills & Technical Stack

• Cloud & Big Data (AWS):

• Storage & Compute: Amazon S3, AWS Lambda, Amazon EMR, Amazon Kinesis.
• Analytics & Warehousing: Amazon Athena, Amazon Redshift, AWS Glue, Apache Spark. [1, 2, 3, 4]

• AI & Machine Learning:

• Frameworks & Services: Amazon Bedrock, Amazon SageMaker AI, LLMs, embeddings, vector search, and RAG implementation patterns. [1]

• Programming & Scripting:

• Strong proficiency in

  Python

  and

  SQL

  .
• Experience with transformation and orchestration tools like

  dbt

  ,

  Apache Airflow

  , or

  Dagster

  . [1, 2, 3]

• Infrastructure & DevOps:

• Infrastructure as Code (IaC) using

  Terraform

  ,

  AWS CDK

  , or

  CloudFormation

  , paired with CI/CD pipelines. [1]

Qualifications & Experience

• Education:

  Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related quantitative field. [1, 2]

• Experience:

  3+ to 5+ years of hands-on experience in data engineering, cloud architecture, or analytics, with a growing portfolio of applied AI/ML or Generative AI projects. [1, 2]