Senior Data Engineer

We are seeking a Data Engineer with strong expertise in Data Engineeringincluding data modeling, ETL pipeline development, data governance, and Data Build Tool (DBT)along with proficiency in Python, PySpark, and AWS services such as Glue, Redshift, Lambda, and DynamoDB. Candidates should also have in-depth knowledge of MySQL, including core and advanced concepts.

• Proficiency with Python and SQL for data processing (spark is a bonus)
• Working experience with AWS storage and database/data warehouse services - s3, Redshift, RDS, DynamoDB etc.
• Hands on experience with AWS compute services such as EC2, ECR, Lambda (layers/triggers) etc.
• Ensure proper logging, error handling, and performance monitoring using CloudWatch.
• Design and develop ETL Pipelines using s3, AWS Glue (crawlers, ETL, database), Lambda and Athena
• Working experience with AWS Identity Management services - IAM (Role creation; customer managed/inline policies), know how to setup IAM roles to communicate between several AWS services
• Build and maintain RESTful APIs using API Gateway, Lambda and Python to establish connection between front-end and back-end (basically construct API and methods to trigger Lamba functions)
• Knowledge of developing AWS CloudFormation templates (json/yaml) for deploying AWS services & CI/CD and Git skills covering Github