Qentelli logo

AWS Data Engineer (Gen AI)

Qentelli

6 - 11 years

Hyderabad

Posted:6 days ago

Job Description

Job Summary:

We are looking for an experienced AWS Data Engineer with strong hands-on experience in Python, SQL, AWS data services, ETL/ELT pipelines, and data warehousing. The candidate should have experience building scalable data pipelines, processing structured and unstructured data, and experienced in Generative AI applications.

Roles & Responsibilities:

  • Design, develop, and maintain scalable ETL/ELT data pipelines on AWS.
  • Work with AWS services such as S3, Glue, Redshift, Athena, Lambda, EMR, and Step Functions.
  • Develop Python scripts and optimize SQL queries for data processing.
  • Build and maintain data lakes, data warehouses, and data models.
  • Process structured and unstructured data from databases, APIs, and files.
  • Implement batch and incremental data processing.
  • Perform data validation, quality checks, monitoring, and troubleshooting.
  • Optimize pipeline performance, reliability, and cost efficiency.
  • Implement AWS security, IAM, encryption, and access controls.
  • Support data preparation and processing for Generative AI applications.
  • Work with documents, embeddings, vector databases, and AI-powered search applications.
  • Collaborate with data scientists, developers, analysts, and business teams.
  • Maintain Git, CI/CD, technical documentation, and production support processes.

Mandatory Skills:

AWS Data Engineering

  • 6+ years of Data Engineering experience.
  • Strong hands-on experience with AWS S3, Glue, Redshift, Athena, Lambda, and IAM.
  • Experience developing ETL/ELT data pipelines.
  • Strong knowledge of data lakes and data warehousing.
  • Experience handling large datasets, batch processing, and incremental loading.
  • Experience with pipeline monitoring and troubleshooting.

Programming & Database

  • Strong Python programming skills.
  • Strong SQL skills, including Joins, Subqueries, CTEs, Window Functions, and Query Optimization.
  • Experience with PostgreSQL, MySQL, Oracle, or SQL Server.
  • Knowledge of data modeling and database design.
  • Experience working with JSON, CSV, and Parquet formats.

Engineering Practices

  • Experience with Git and source-code management.
  • Knowledge of CI/CD processes.
  • Good understanding of AWS security and IAM.
  • Experience with logging, monitoring, error handling, and production support.
  • Strong debugging and problem-solving skills.

Preferred Skills

  • Experience with Generative AI, Amazon Bedrock, and AWS AI services.
  • Knowledge of RAG, embeddings, vector databases, and Amazon OpenSearch.
  • Experience with document processing (PDF, Word, HTML, XML).
  • Experience with PySpark, Spark, and AWS EMR.
  • Knowledge of Apache Airflow or similar orchestration tools.
  • Experience with Terraform, CloudFormation, Docker, and REST APIs.

Skills Required

Artificial Intelligence Data Python AWS Amazon Ec2 Athena Gen

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Work from Office

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Artificial Intelligence Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Qentelli logo
Qentelli

RecommendedJobs for You

Qentelli logo

Hyderabad

Qentelli logo

Hyderabad