Hadoop

As a Hadoop professional with 4 to 6 years of relevant experience, you will be responsible for developing data processing solutions using Hadoop components like HDFS, YARN, and Map Reduce. Your expertise in PySpark for distributed data processing, transformation, and analytics in Hadoop environments will be crucial. Additionally, you will utilize Hive for querying, data warehousing, and managing structured data on large-scale datasets. Your role will involve creating and optimizing Hadoop-based applications to ensure performance, scalability, and reliability. Familiarity with ETL processes, data ingestion techniques, and workflows within Hadoop ecosystems will be essential. You will also work with job scheduling and workflow management tools such as Apache Oozie or similar to streamline processes. Troubleshooting Hadoop environment issues and optimizing PySpark jobs for efficient processing will be part of your responsibilities. Strong teamwork, collaboration, and communication skills are required for effective cross-functional team collaboration. **Key Responsibilities:** - Develop data processing solutions using Hadoop components like HDFS, YARN, and Map Reduce - Utilize PySpark for distributed data processing, transformation, and analytics - Utilize Hive for querying, data warehousing, and managing structured data - Create and optimize Hadoop-based applications for performance, scalability, and reliability - Implement ETL processes, data ingestion techniques, and workflows within Hadoop ecosystems - Work with job scheduling and workflow management tools such as Apache Oozie - Troubleshoot Hadoop environment issues and optimize PySpark jobs for efficiency - Collaborate effectively in cross-functional teams **Qualifications Required:** - 4 to 6 years of experience working with Hadoop and big data technologies - Proficiency in PySpark for distributed data processing - Expertise in Hive for querying and data warehousing - Familiarity with job scheduling and workflow management tools - Strong understanding of data partitioning, indexing, and optimization techniques The hiring process for this position includes screening (HR Round), Technical Round 1, Technical Round 2, and Final HR Round. This role is based in Bangalore and Chennai. (Note: The provided job description does not include any additional details about the company.),