CDP Engineer

Job Description

Assistant / Deputy Manager Hyderabad B.E./MCA/B.Tech/M.sc (I.T.) 25-35 Experience & Role: Experience – above 5-8 years relevant experience. Role - We are looking for a technically strong and detail-oriented professional to manage and support our Cloudera Data Platform (CDP) ecosystem. The ideal candidate should possess in-depth expertise in distributed data processing frameworks and hands-on experience with core Hadoop components. This role requires both operational excellence and technical depth, with an emphasis on optimizing data processing pipelines and maintaining high system availability. Job Description: - Administer and maintain the Cloudera Data Platform (CDP) across all environments (dev/test/prod) - Strong expertise in Big Data ecosystem like Spark, Hive, Sqoop, HDFS, Map Reduce, Oozie, Yarn, HBase, Nifi. - Develop and optimize complex Hive queries, including the use of analytical functions for reporting and data transformation. - Create custom UDFs in Hive to handle specific business logic and integration needs. - Ensure efficient data ingestion and movement using Sqoop, Nifi, and Oozie workflows. - Work with various data formats (CSV, TSV, Parquet, ORC, JSON, AVRO) and compression techniques (Gzip, Snappy) to maximize performance and storage. - Monitor and tune performance of YARN and Spark applications for optimal resource utilization. - In depth Knowledge on Architecture of Distributed Systems and Parallel Computing. Internal - Good knowledge in Oracle PL/SQL and shell scripting. - Strong problem-solving and analytical thinking. - Effective communication and documentation skills. - Ability to collaborate across multi-disciplinary teams. - Self-driven with the ability to manage multiple priorities under tight timelines. Job Types: Full-time, Permanent Pay: Up to ₹100,000.00 per year Schedule: Day shift Monday to Friday Work Location: In person