Big Data Operations Engineer

S M Software Solutions Inc

5 years

0 Lacs

hyderabad telangana india

Posted:2 weeks ago| Platform:

Apply

Skills Required

data monitoring development support hadoop storage resolve informatica kafka management reporting analysis documentation hive etl pipeline dashboard kibana linux unix scripting communication coordination spark python gcp aws

Work Mode

On-site

Job Type

Full Time

Job Description

Location :

Onsite, Hyderabad Softility

Experience

: 5+ years

Role Overview

We are looking for a skilled

Big Data Operations Engineer

with hands-on experience in supporting and maintaining large-scale data platforms. The role involves ensuring platform availability, managing incidents, monitoring performance through dashboards, and coordinating with development teams for enhancements. While the primary focus is on operations and support, the candidate should also have exposure to Big Data development concepts and tools.

Key Responsibilities

Monitor and maintain Hadoop clusters, HDFS storage, and YARN resource management.
Troubleshoot and resolve production issues, ensuring minimal downtime and quick recovery.
Support Informatica workflows in production, including error handling and performance tuning.
Manage Kafka topics, producers/consumers, and cluster health monitoring.
Work with Anthos for hybrid cloud platform management and operational tasks.
Support observability and log pipelines using Cribl.
Build and maintain dashboards for monitoring and reporting using tools like New Relic.
Perform incident management, root cause analysis, and preventive action planning.
Collaborate with development teams to roll out new jobs, updates, and fixes.
Maintain operational documentation, runbooks, and knowledge repositories.

Required Skills & Experience

5-6 years of experience in Big Data operations/support.
Strong knowledge of the Hadoop ecosystem, including HDFS, YARN, Hive, and Oozie.
Experience supporting ETL workflows (preferably Informatica).
Familiarity with Kafka operations and troubleshooting.
Exposure to Anthos and Cribl for platform and pipeline monitoring.
Experience with dashboard/monitoring tools (Grafana, Kibana, or equivalent).
Hands-on with Linux/Unix administration, shell scripting, and automation.
Solid experience in incident, change, and problem management.
Good communication and coordination skills for working with cross-functional teams.

Nice-to-Have

Exposure to Big Data development (Hive, Spark, Python, or SQL).
Knowledge of cloud platforms (GCP/AWS/Azure).
Familiarity with Cloudera Manager or other cluster monitoring tools.

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

S M Software Solutions Inc

Login to

Please Verify Your Phone or Email

Confirm Action

Big Data Operations Engineer