Data Engineer ( AWS Platforms)

Job

We are looking for an experienced Data Engineer who is proficient in designing and building data pipelines, data platforms, and data analysis solutions on the AWS ecosystem. The successful candidate will be the single point of contact for all data-related queries and pipeline delivery for the application products.

Key Responsibilities:

• Design, build and maintain data pipelines and data platforms on AWS components, including AWS Data Lake, AWS Athena, Kafka, AWS Kinesis, AWS Aurora, AWS Document DB, Apache NiFi, Apache Spark, Apache Zookeeper, AWS S3, AWS EMR, and AWS Glue.
• Act as the single source of truth for all DBAT product data and handle both structured and unstructured data sources.
• Collaborate with the BIU team to provide detailed analysis of products and handle various logs flowing from different products.
• Work with AI/ML solutions to provide the required data and handle all data requirements.
• Create a data dictionary of all different data sources to ensure consistency and accuracy.
• Stay up-to-date with the latest trends and best practices in data engineering and AWS components.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
• 3+ years of experience in data engineering, ETL development, data modelling, and data architecture.
• Strong experience with AWS components such as AWS Data Lake, AWS Athena, Kafka, AWS Kinesis, AWS Aurora, AWS Document DB, Apache NiFi, Apache Spark, Apache Zookeeper, AWS S3, AWS EMR, and AWS Glue.
• Proficiency in programming languages such as Python, SQL, Pyspark etc. Familiarity with data warehousing concepts, relational databases, and NoSQL databases.
• Strong analytical, problem-solving, and communication skills.
• Ability to work independently and collaboratively in a fast-paced environment.