AWS Data /API Gateway Pipeline Engineer

Job Title:

Experience:

Location:

AWS Data/API Gateway Pipeline Engineer responsible for designing, building, and maintaining real-time, serverless data pipelines and API services. This role requires extensive hands-on experience with Java, Python, Redis, DynamoDB Streams, and PostgreSQL, along with working knowledge of AWS Lambda and AWS Glue for data processing and orchestration.

This position involves collaboration with architects, backend developers, and DevOps engineers to deliver scalable, event-driven data solutions and secure API services across cloud-native systems.

Key Responsibilities:

API & Backend Engineering

• Build and deploy RESTful APIs using AWS API Gateway, Lambda, and Java and Python.
• Integrate backend APIs with Redis for low-latency caching and pub/sub messaging.
• Use PostgreSQL for structured data storage and transactional processing.
• Secure APIs using IAM, OAuth2, and JWT, and implement throttling and versioning strategies.

Data Pipeline & Streaming

• Design and develop event-driven data pipelines using DynamoDB Streams to trigger downstream processing.
• Use AWS Glue to orchestrate ETL jobs for batch and semi-structured data workflows.
• Build and maintain Lambda functions to process real-time events and orchestrate data flows.
• Ensure data consistency and resilience across services, queues, and databases.

Cloud Infrastructure & DevOps

• Deploy and manage cloud infrastructure using CloudFormation, Terraform, or AWS CDK.
• Monitor system health and service metrics using CloudWatch, SNS and structured logging.
• Contribute to CI/CD pipeline development for testing and deploying Lambda/API services.

Required Skills and Experience

• Bachelor's degree in computer science, Engineering, or a related field.
• Over 6 years of experience in developing backend or data pipeline services using Java and Python.
• Strong hands-on experience with:

• AWS API Gateway, Lambda, DynamoDB Streams

• Redis

  (caching, messaging)

• PostgreSQL

  (schema design, tuning, SQL)

• AWS Glue

  for ETL jobs and data transformation
• Solid understanding of REST API design principles, serverless computing, and real-time architecture.

Preferred Skills and Experience

• Familiarity with Kafka, Kinesis, or other message streaming systems
• Swagger/OpenAPI for API documentation
• Docker and Kubernetes (EKS)
• Git, CI/CD tools (e.g., GitHub Actions)
• Experience with asynchronous event processing, retries, and dead-letter queues (DLQs)
• Exposure to data lake architectures (S3, Glue Data )