Data Engineer

Job Title :

Location:

Experience:

Role Overview

d Data Engine

Key Responsibilities:

• Design, develop, and maintain scala

  ble big data pipel

  ines and architectures to support business and analytical needs
• Build and optim

  ize data processing sys

  tems using technologies such

  as Hadoop, Hive, Spark, and Python/Java

• Implem

  ent data governance princi

  ples, ensuring data quality, integrity, and security across all data layers.
• Collaborate w

  ith cross-functional t

  eams to understand business requirements and translate them into technical solutions.
• Develop and man

  age data workflows and orchestra

  tion using tools l

  ike Airflow, Azkaban, or Luigi

• Integrate and manage data from diverse sources includ

  ing SQL, NoSQL datab

  ases, APIs, and streaming data.
• Design and maint

  ain real-time data streaming and message queuing sys

  tems (e.g., Kafka, Spark Streaming, Storm).
• Support the deployment and monitoring of data systems

  on cloud platf

  orms such

  as AWS or Azure

  .
• Cond

  uct data profiling, cleansing, and valida

  tion to ensure accuracy and reliability.
• Continuously improve system performance, scalability, and efficiency through automation and optimizat

  ion.

Qualification:

• Bachelor’s or Graduate

  degree in Computer Science, Statistics, Informatics, Information Systems, or a related quantitative field

  .

• 4+ years of hands-on expe

  rience as a Data Engineer or in a related data engineering
• role.
• Proven expertise

  with big data techno

  logies such as Hadoop, Hive, Spark, and Kafka.
• Strong programming skil

  ls in Python, Java, or Scala

• Proficiency

  with SQL and NoSQL dat

  abases (e.g., MySQL, PostgreSQL, Cassandra).
• Experience

  with data pipeline and workflow management

  tools such as Airflow, Azkaban, or Luigi.
• Familiarity

  with cloud data se

  rvices on AWS or Azure (e.g., EC2, EMR, RDS, Redshift, HDInsights, DataLake, DataBricks).
• Working knowled

  ge of stream-processing fram

  eworks such as Spark Streaming or Storm.
• Strong analytical and problem-solving
  skills with the ability to design solutions for complex business use cases.

• Excellent project management and organizational

  skills with experience working in dynamic, cross-functional environments.