Technology & Transformation- EAD- Engineering- AWS Data Engineer (Consultant/Senior Consultant)

Key Responsibilities:

1. Design and implement scalable, high-performance data pipelines using AWS services

2. Develop and optimize ETL processes using AWS Glue, EMR, and Lambda

3. Build and maintain data lakes using S3 and Delta Lake

4. Create and manage analytics solutions using Amazon Athena and Redshift

5. Design and implement database solutions using Aurora, RDS, and DynamoDB

6. Develop serverless workflows using AWS Step Functions

7. Write efficient and maintainable code using Python/PySpark, and SQL/PostgrSQL

8. Ensure data quality, security, and compliance with industry standards

9. Collaborate with data scientists and analysts to support their data needs

10. Optimize data architecture for performance and cost-efficiency

11. Troubleshoot and resolve data pipeline and infrastructure issues

Required Qualifications:

1. bachelor's degree in computer science, Information Technology, or related field

2. Relevant years of experience as a Data Engineer, with at least 60% of experience focusing on AWS

AWS data services: Glue, EMR, Lambda, Athena, Redshift, S3

4. Experience with data lake technologies, particularly Delta Lake

Aurora, RDS, DynamoDB, PostgreSQL

Python and PySpark

PostgreSQL

8. Experience with AWS Step Functions for workflow orchestration

9. Familiarity with data modeling and schema design

10. Knowledge of data security and compliance requirements

11. Excellent problem-solving and analytical skills

12. Strong communication and collaboration abilities

Preferred Qualifications:

1. AWS Certified Data Analytics - Specialty

2. AWS Certified Solutions Architect - Associate or Professional

3. Experience with real-time data processing using Kinesis or Kafka

4. Knowledge of machine learning workflows on AWS (e.g., SageMaker)

5. Familiarity with containerization technologies (Docker, Kubernetes)

6. Experience with CI/CD pipelines and infrastructure-as-code (e.g., CloudFormation, Terraform)

Technical Skills:

- AWS Services: Glue, EMR, Lambda, Athena, Redshift, S3, Aurora, RDS, DynamoDB, Step Functions

- Big Data: Hadoop, Spark, Delta Lake

- Programming: Python, PySpark

- Databases: SQL, PostgreSQL, NoSQL

- Data Warehousing and Analytics

- ETL/ELT processes

- Data Lake architectures

- Version control: Git

- Agile methodologies