Hadoop Maintenance Engineer

Job Title: Hadoop Maintenance Engineer

Location: Mohali
Experience: 2–5 Years

Job Overview:

We are seeking a skilled Hadoop Maintenance Engineer to manage, monitor, and optimize our Hadoop ecosystem. The ideal candidate will have hands-on experience in maintaining HDFS, Hive, and Spark clusters, ensuring smooth data flow and system reliability across distributed environments.

Key Responsibilities:

• Perform daily monitoring, maintenance, and troubleshooting of Hadoop clusters (HDFS, Hive, Spark).
• Manage cluster upgrades, patching, and performance tuning.
• Ensure data reliability, availability, and security within the Hadoop ecosystem.
• Diagnose and resolve cluster issues, node failures, and performance bottlenecks.
• Implement data replication, backup, and recovery strategies.
• Work closely with data engineering teams to optimize Hive queries and Spark jobs.
• Monitor and maintain system health using tools like Ambari, Cloudera Manager, or similar.
• Develop scripts for automation and system improvements (using Shell, Python, etc.).
• Document processes, configurations, and incident resolutions.

Required Skills & Experience:

• 2–5 years of experience in Hadoop ecosystem management.
• Strong knowledge of HDFS architecture and file management.
• Expertise in Hive (data warehousing, query optimization) and Spark (batch/stream processing).
• Proficiency in Linux/Unix systems and shell scripting.
• Familiarity with Hadoop ecosystem tools like YARN, Oozie, Sqoop, or Kafka (preferred).
• Understanding of cluster resource management and tuning.
• Experience with monitoring tools and log analysis.

Preferred Qualifications:

• Experience with Cloudera or Hortonworks distributions.
• Knowledge of cloud-based Hadoop deployments (AWS EMR, Azure HDInsight, GCP Dataproc).
• Strong analytical and troubleshooting skills.
• Excellent communication and documentation abilities.

Education:

• Bachelor’s degree in Computer Science, Information Technology, or related field.

Job Types: Full-time, Permanent

Work Location: In person