Data Engineer

Title :

Work Mode :

Experience :

Location :

Notice Period :

Data Engineer

Key Responsibilities:

• Design, develop, and maintain large-scale

  data pipelines

  and

  ETL workflows

  using Big Data technologies.
• Build and optimize

  data processing frameworks

  on

  AWS

  using services like

  S3, EMR, Glue, Redshift, Lambda

  , etc.
• Work on data ingestion, transformation, and storage solutions for structured and unstructured datasets.
  
• Ensure high performance, scalability, and reliability of data systems.
  
• Collaborate with cross-functional teams to translate business requirements into technical data solutions.
  
• Implement best practices for

  data quality

  ,

  governance

  , and

  security

  .
• Monitor, troubleshoot, and optimize existing data pipelines for efficiency and cost-effectiveness.
  

Mandatory Skills:

• Big Data Technologies:

  Hadoop, Spark, Hive, HDFS, Kafka, etc.

• Cloud Platform:

  AWS (Amazon Web Services) experience with S3, EMR, Glue, Redshift, Lambda, etc.

• Programming:

  Strong hands-on experience in

  PySpark

  or

  Java

  or

  Scala

  (any one is mandatory).

• Data Modeling

  and

  ETL Development

  experience.

• SQL

  and performance tuning for large datasets.
• Strong analytical, debugging, and problem-solving skills.
  

Good to Have Skills:

• Experience with CI/CD, Airflow, or data orchestration tools.
  
• Exposure to containerization (Docker, Kubernetes).
  
• Knowledge of data lake and data warehouse concepts.
  
• Familiarity with monitoring tools and cost optimization on AWS.