Hiring Bigdata SME - Chennai/ Bangalore

Job Title: Data Engineering Hadoop SME

Experience:

Job Summary

Hadoop/Data Engineering SME

Key Responsibilities

• Act as the

  SME for Hadoop and Big Data engineering

  solutions.
• Design and develop highly scalable

  ETL/data pipelines

  for large-volume data processing.
• Provide technical design and solution guidance using

  Hadoop, HDFS, Spark, Hive, and related technologies

  .
• Develop and optimize complex

  Spark/PySpark/Scala

  processing jobs.
• Build and manage workflow orchestration using

  Airflow

  .
• Write and optimize complex

  SQL/HiveQL

  queries.
• Drive

  performance tuning

  and troubleshoot complex production issues involving Spark, Hive, and distributed workloads.
• Design ingestion solutions for structured and unstructured data using

  Kafka, APIs, files, and other sources

  .
• Provide expertise in

  data modelling, data quality, lineage, and governance

  .
• Conduct technical reviews, code reviews, and enforce engineering best practices.
• Mentor developers and provide technical guidance to engineering teams.
• Collaborate with architects, business stakeholders, and upstream/downstream teams on solution design.
• Create and maintain technical design and architecture documentation.

Mandatory Skills

• 10+ years

  of Data Engineering/Big Data experience, with strong hands-on development expertise.
• Expert-level knowledge of

  Hadoop ecosystem and HDFS

  .
• Strong hands-on experience with

  Apache Spark / PySpark

  .
• Strong knowledge of

  Hive / HiveQL

  .
• Strong programming skills in

  Python and/or Scala

  .
• Advanced

  SQL

  and strong

  data modelling

  fundamentals.
• Hands-on experience with

  Airflow

  or similar orchestration platforms.
• Strong experience in

  performance tuning and troubleshooting Spark/Hadoop workloads

  .
• Good understanding of

  distributed computing and large-scale data processing

  .
• Linux and Shell scripting knowledge.
• Experience with Git and Agile development practices.