Aws Data Engineer | MNC | PAN India

Experience:

Role Overview

AWS Data Engineer

Key Responsibilities

Data Engineering & Pipeline Development

• Design and develop scalable

  ETL/ELT pipelines

  using

  AWS, PySpark, and Databricks

  .
• Build and optimize

  data ingestion frameworks

  , ensuring high performance and reliability.
• Implement

  batch and streaming pipelines

  using Spark/PySpark.

Cloud & Platform Engineering

• Work extensively on

  AWS services

  such as S3, Lambda, Glue, EMR, Athena, Step Functions, Redshift, IAM, CloudWatch.
• Develop and maintain

  Databricks notebooks

  , jobs, workflows, and clusters.

Data Modelling & SQL

• Create and optimize

  complex SQL queries

  , stored procedures, and transformations.
• Design

  data models

  (Dimensional, Star/Snowflake schemas) for analytics & BI needs.

Performance, Testing & Deployment

• Optimize Spark and SQL jobs for

  cost, performance, and scalability

  .
• Conduct

  unit testing, integration testing

  , and ensure quality delivery.
• Work with DevOps teams to deploy pipelines using

  CI/CD

  tools (Git, Jenkins, CodePipeline).

Collaboration & Stakeholder Management

• Work closely with Data Architects, Analysts, and Business Teams to translate requirements into technical solutions.
  
• Provide

  technical guidance

  and best practices to junior team members.

Required Skills

• Strong hands-on experience with AWS Cloud

  (Glue, S3, EMR, Lambda, Redshift, Athena).

• Expertise in Databricks

  (Notebooks, Delta Lake, Spark jobs, Workflows).

• Advanced SQL skills

  for large datasets.

• Strong PySpark development experience

  (Transformations, Actions, Optimization).
• Experience with

  Delta Lake, Lakehouse architecture

  , and distributed computing.
• Knowledge of

  version control (Git)

  and CI/CD pipelines.
• Solid understanding of

  data warehousing

  and data governance concepts.