AWS Data Engineer

Job Title: Senior Data Engineer

Key Responsibilities

As a Senior Data Engineer, you will:

• Data Pipeline Development:

  Design, build, and maintain scalable data pipelines using PySpark and Python.

• AWS Cloud Integration:

  Work with AWS cloud services (S3, Lambda, Glue, EMR, Redshift) for data ingestion, processing, and storage.

• ETL Workflow Management:

  Implement and maintain ETL workflows using DBT and orchestration tools (e.g., Airflow).

• Data Warehousing:

  Design and manage data models in Snowflake, ensuring performance and reliability.

• SQL Optimization:

  Utilize SQL for querying and optimizing datasets across different databases.

• Data Integration:

  Integrate and manage data from MongoDB, Kafka, and other streaming or NoSQL sources.

• Collaboration & Support:

  Collaborate with data scientists, analysts, and other engineers to support advanced analytics and Machine Learning (ML) initiatives.

• Data Quality & Governance:

  Ensure data quality, lineage, and governance through best practices and tools.

Mandatory Skills & Experience

• Strong programming skills in

  Python

  and

  PySpark

  .
• Hands-on experience with

  AWS data services

  (S3, Lambda, Glue, EMR, Redshift).
• Proficiency in

  SQL

  and experience with

  DBT

  for data transformation.
• Experience with

  Snowflake

  for data warehousing.
• Knowledge of

  MongoDB

  ,

  Kafka

  , and data streaming concepts.
• Good understanding of

  data architecture, data modeling, and data governance

  .
• Familiarity with large-scale data platforms.

Essential Professional Skills

• Excellent

  problem-solving skills

  .
• Ability to work

  independently or as part of a team

  .
• Experience with

  CI/CD

  and

  DevOps practices

  in a data engineering environment (Plus).

Qualifications

• Proven hands-on experience working with

  large-scale data platforms

  .
• Strong background in

  Python, PySpark, AWS

  , and modern data warehousing tools such as

  Snowflake and DBT

  .
• Familiarity with

  NoSQL databases

  like MongoDB and real-time streaming platforms like Kafka.