GPU Infrastructure & Data Center Engineer

About the Role

We are seeking a highly skilled IT Solutions & GPU Infrastructure Lead to take complete ownership of our GPU-based server infrastructure. This role focuses on next-generation GPU systems used for AI/ML workloads, covering every aspect from data center colocation and setup to GPU slicing, MIG management, resource allocation, optimization, and compliance. You will lead the end-to-end lifecycle of GPU infrastructure — ensuring all servers are optimized, secure, and production-ready for both internal and customer use.

Key Responsibilities

1. Colocation & Infrastructure Setup

GPU colocation and end-to-end infrastructure setup will be entirely under your ownership and responsibility.

• Coordinate with data centers for rack installation, power, and cooling.
• Deploy and configure GPU-based servers for production readiness.

2. GPU & AI/ML Infrastructure

• Manage GPU slicing and MIG (Multi-Instance GPU) for multi-tenant workloads.
• Install and maintain the NVIDIA software stack — CUDA, cuDNN, NCCL, and DCGM.
• Optimize GPU infrastructure for AI/ML workloads (TensorFlow, PyTorch, RAPIDS).
• Support multi-GPU scaling using NVLink and PCIe passthrough.

3. Systems & Virtualization

• Administer Linux-based environments (Ubuntu, CentOS, Rocky) along with other environments.
• Manage virtualization platforms such as VMware, KVM, or Proxmox with GPU passthrough.
• Handle container orchestration with Docker and Kubernetes GPU Operators.
• Integrate high-performance storage (NFS, Ceph, SAN/NAS) for large-scale datasets.

4. Monitoring & Performance Optimization

• Monitor GPU and system performance using Prometheus, Grafana, NVIDIA DCGM, and nvidia-smi.
• Proactively detect, analyze, and resolve GPU or system bottlenecks.
• Optimize GPU nodes for training and inference performance.
• Implement structured logging, alerts, and usage reporting.
• one should have to administer, manage, monitor and maintain GPU infrastructure for AI workloads.

5. Security & Compliance

• Harden GPU servers for multi-tenant workloads.
• Manage driver, firmware, and software license compliance.
• Ensure infrastructure security and audit readiness with periodic patching and updates.

6. Networking & High-Performance I/O

• Configure and maintain high-speed network fabrics (InfiniBand, RDMA, RoCE).
• Optimize low-latency interconnects for distributed GPU workloads.
• Troubleshoot and enhance data transfer performance.

7. Customer & Infrastructure Ownership

• Serve as the primary contact for GPU resource allocation.
• Provision GPU slices or MIG instances for internal and external teams.
• Troubleshoot, document, and optimize workload performance.

Qualifications

• Proven experience in data center server setup and colocation.
• Deep expertise in GPU server administration (NVIDIA A100/H100 or equivalent).
• Strong working knowledge of GPU slicing, MIG, CUDA, NCCL, and NVIDIA drivers.
• Experience with Linux administration, virtualization (VMware/KVM/Proxmox), and containers (Docker/Kubernetes).
• Hands-on experience with AI/ML frameworks such as TensorFlow and PyTorch.
• Familiarity with monitoring tools (Prometheus, Grafana, DCGM).
• Knowledge of storage systems (NFS, Ceph) and high-performance networking.
• Strong vendor coordination and infrastructure management skills.

Why This Role Matters

This position owns the entire lifecycle of GPU-based infrastructure — from colocation to slicing, monitoring, and optimization. You will build and maintain the backbone of our AI/ML infrastructure, ensuring that all systems are efficient, scalable, and production-grade.