AVP - Application Engineering/Engineering Manager

location: On-Premises Infrastructure (India-based preferred)

Department: Technology Operations / Platform Engineering

Reporting To: Engineering Manager / AVP - Application Engineering

Job Summary:

We are seeking a skilled and experienced Technology Stack Administrator to manage and optimize our on-premises infrastructure comprising RedHat OpenShift, Apache Spark, Apache Airflow, Delta Lake, Apache Kafka, and Debezium. The ideal candidate will have hands-on experience in deploying, configuring, monitoring, and troubleshooting these platforms, ensuring high availability, performance, and security.

Key Responsibilities:

• Platform Administration:

• Install, configure, and maintain 

  RedHat OpenShift clusters

   in an on-premises environment.
• Build and orchestrate data pipelines using Apache Airflow.
• Implement and manage Delta Lake for ACID-compliant data lake architecture.
• Manage 

  Apache Spark

   clusters for distributed data processing.
• Administer and maintain Apache Spark clusters, ensuring optimal performance and

resource utilization

• Develop real-time data streaming and CDC solutions using Apache Kafka and Debezium.
• Build and deploy containerized applications on RedHat OpenShift.
• Integrate and manage Unity Catalog for centralized data governance, access control, and lineage tracking.
• Integrate platform components with Nexus Repository Manager for artifact lifecycle management and CI/CD workflows.Monitoring & Optimization
• Set up monitoring tools and dashboards (e.g., Prometheus, Grafana, ELK).
• Optimize performance and resource utilization across platforms.
• Conduct regular health checks and capacity planning.

• Security & Compliance:

• Implement role-based access control (RBAC), TLS encryption, and audit logging.
• Ensure compliance with internal and external data governance policies.

• Automation & CI/CD:

• Develop and maintain automation scripts using Ansible, Terraform, or similar tools.
• Integrate platforms with CI/CD pipelines for seamless deployment and updates.
• Maintain version control and artifact repositories via Nexus.

• Troubleshooting & Support:

• Provide L2/L3 support for platform-related issues.
• Collaborate with development and data engineering teams to resolve integration challenges.

Required Skills & Experience:

• 6–9 years of experience in administering enterprise-grade technology stacks.
• Strong expertise in 

  OpenShift

  , 

  Spark

  , 

  Airflow

  , 

  Kafka

  , and 

  Debezium

  .
• Experience with 

  Linux system administration

   and 

  networking fundamentals

  .
• Familiarity with 

  containerization (Docker)

   and 

  orchestration (Kubernetes)

  .
• Proficiency in scripting languages (Bash, Python).
• Understanding of 

  data lake architectures

   and 

  streaming data pipelines

  .
• Unity Catalog
• Nexus Repository Manager

Preferred Qualifications:

• RedHat Certified Specialist in OpenShift Administration.
• Experience with 

  Hadoop ecosystem

  , 

  Hive

  , or 

  Presto

   is a plus.
• Exposure to 

  cloud-native tools

   (even if deployed on-premise).
• Knowledge of 

  DevOps practices

   and 

  GitOps workflows

  .

Soft Skills:

• Strong analytical and problem-solving skills.
• Excellent communication and documentation abilities.
• Ability to work independently and in cross-functional teams.

Job Types:

Work Location