Hadoop Developer

Big Data Hadoop Developer

As a Big Data Hadoop Developer, you will work closely with data architects, analysts, and business stakeholders to build reliable and high-performance data solutions. You will be involved in processing large datasets using Hadoop, Spark, and Hive while ensuring data quality, availability, and performance.

Key Responsibilities:

• Design, develop, and maintain scalable big data systems and data pipelines.

• Implement data processing frameworks using Hadoop, Spark, and Hive.

• Develop and optimize ETL workflows to handle large-scale structured and unstructured data.

• Perform data validation, quality checks, and troubleshooting for pipeline reliability.

• Optimize data storage, partitioning, and performance across the Hadoop ecosystem.

• Collaborate with cross-functional teams to translate business requirements into technical solutions.

• Ensure security, governance, and compliance standards are followed in big data environments.

• Monitor and fine-tune Spark and Hadoop jobs for performance improvements.

Required Skills & Qualifications:

• 6+ years of hands-on experience in Big Data technologies.

• Strong expertise in Hadoop ecosystem: HDFS, YARN, MapReduce.

• Proficiency in Apache Spark (batch and streaming) and Hive.

• Experience in designing and implementing ETL pipelines.

• Strong understanding of performance tuning, partitioning, and optimization techniques.

• Experience with Python, Scala, Java, or Shell scripting.

Preferred Skills:

• Exposure to real-time processing tools like Kafka, Flink, or Storm.

• Experience with NoSQL databases such as HBase, Cassandra, or MongoDB.

• Familiarity with data governance and security practices.

This role requires working from the Hyderabad(or) Vijayawada office, Monday to Friday (5 days WFO).