SRE - Hadoop

Job Description

JOB DESCRIPTION Minimum of 1 year of experience in Linux/Unix Administration. Minimum of 2 years of hands on experience with managing infra on public cloud i.e Azure/AWS/GCP Over 4+ years of experience in Hadoop administration. Strong understanding of networking, open-source technologies, and tools. Familiar with best practices and IT operations for maintaining always-up, always-available services. Experience and participation during on-call rotation. Excellent communication skills. Solid expertise in Linux networking, including IP, iptables, and IPsec.Proficient in scripting and coding with languages such as Perl, Golang, or Python. Strong Knowledge of databases like Mysql,Nosql,Sql serverHand on experience with setting up , configuring and Managing Nginx as reverse proxy and load balancing in high traffic environments. Hands-on experience with both private and public cloud environments. Strong troubleshooting skills and operational expertise in areas such as system capacity, bottlenecks, memory, CPU, OS, storage, and networking. Practical experience with the Hadoop stack, including Hdfs,HBase,Hive, Pig, Airflow, YARN, HDFS, Ranger, Kafka, and Druid. Good to have experience with Design,develop and maintain Airflow DAGs and tasks to automate BAU processes,ensuring they are robust,scalable and efficient. Good to have experience with ELK stack administration. Experience in administering Kerberos and LDAP. Familiarity with open-source configuration management and deployment tools like Puppet, Salt, or Ansible.Responsible for the implementation and ongoing administration of Hadoop infrastructure. Experience in capacity planning and performance tuning of Hadoop clusters. Collaborate effectively with infrastructure, network, database, application, and business intelligence teams to ensure high data quality and availability. Develop tools and services to enhance debuggability and supportability.Work closely with security teams to apply Hadoop updates, OS patches, and version upgrades as needed. Troubleshoot complex production issues, identify root causes, and provide mitigation strategies. Work closely with teams to optimize the overall performance of the PhonePe Hadoop ecosystem. Experience with setting up & managing monitoring stack like OpenTsdb,Prometheus,ELK,Grafana,Loki