Job Title: GPU DevOps / Infrastructure Setup Engineer
Location: Chennai, India
Experience: 3+ Years
Required GPU Experience: 1–2+ Years of Hands-on Experience
Job Category: DevOps / Infrastructure Engineering / AI Infrastructure
Job Overview
We are looking for an experienced GPU DevOps / Infrastructure Setup Engineer to build, configure, deploy, and maintain GPU-enabled infrastructure supporting AI/ML, Large Language Model (LLM), and High-Performance Computing (HPC) workloads.
The ideal candidate should have practical experience in GPU server configuration, Linux administration, NVIDIA GPU drivers, CUDA, containerization, infrastructure automation, and GPU workload deployment.
Experience with NVIDIA Base Command Manager (BCM), GPU cluster deployment, InfiniBand networking, and production LLM serving will be an added advantage.
Candidates must have hands-on experience with at least one real-world GPU infrastructure setup, GPU cluster deployment, or AI/LLM infrastructure project. They should be able to explain the hardware used, their individual responsibilities, deployment process, and technical challenges resolved.
Key Responsibilities
- Set up, configure, and maintain GPU-enabled servers, compute nodes, and AI infrastructure.
- Deploy and maintain on-premises GPU infrastructure and AI/HPC compute clusters.
- Perform end-to-end GPU cluster deployment, including provisioning, configuration, validation, and testing.
- Install, configure, and troubleshoot Linux operating systems, NVIDIA GPU drivers, CUDA Toolkit, and supporting libraries.
- Validate GPU hardware detection, driver functionality, CUDA compatibility, and GPU workload execution.
- Configure and manage GPU-enabled Docker containers and deploy GPU workloads using Kubernetes.
- Deploy and maintain LLM applications using vLLM, Ollama, Hugging Face Text Generation Inference (TGI), NVIDIA Triton, or similar frameworks.
- Automate infrastructure provisioning and configuration using Bash, Python, Ansible, Terraform, or equivalent tools.
- Develop and maintain CI/CD pipelines for infrastructure and application deployments.
- Monitor GPU utilization, memory consumption, temperature, power usage, hardware health, and workload performance.
- Implement monitoring, logging, alerting, and health checks using Prometheus, Grafana, NVIDIA SMI, DCGM, or equivalent tools.
- Troubleshoot GPU memory errors, driver conflicts, CUDA initialization failures, container issues, networking problems, and workload failures.
- Optimize GPU utilization, inference latency, throughput, VRAM consumption, and infrastructure performance.
- Perform infrastructure maintenance, patching, upgrades, capacity planning, and resource scaling.
- Maintain technical documentation, deployment procedures, configuration guides, and troubleshooting records.
- Collaborate with AI/ML, development, platform, and infrastructure teams to support reliable production deployments.
Required Technical SkillsGPU Hardware and Infrastructure
- Hands-on experience configuring and maintaining GPU-enabled servers.
- Knowledge of GPU server architecture, PCIe, power requirements, cooling, and hardware compatibility.
- Experience with NVIDIA A100, H100, H200, L40S, RTX series, or equivalent GPU platforms.
- Ability to validate GPU hardware detection and troubleshoot hardware-related issues.
Linux Administration, NVIDIA Drivers, and CUDA
- Strong Linux administration and troubleshooting skills.
- Experience installing and configuring NVIDIA GPU drivers and CUDA Toolkit.
- Ability to validate GPU availability using nvidia-smi.
- Experience troubleshooting driver compatibility, CUDA initialization, and GPU workload execution issues.
- Experience running GPU workloads directly on bare-metal infrastructure is preferred.
AI Infrastructure and HPC
- Experience building, configuring, or maintaining GPU-enabled AI infrastructure.
- Understanding of compute nodes, cluster provisioning, and GPU cluster validation.
- Familiarity with NVIDIA Base Command Manager (BCM) is preferred.
- Experience deploying and managing GPU clusters without BCM is also valuable.
- Knowledge of InfiniBand, RDMA, and high-speed networking is an advantage.
- Exposure to multi-GPU or multi-node deployments is desirable.
DevOps, Containers, and Automation
- Hands-on experience with Docker and GPU-enabled containers.
- Kubernetes experience for GPU workload orchestration.
- Scripting and automation using Bash, Python, Ansible, Terraform, or equivalent tools.
- Familiarity with Git, CI/CD pipelines, configuration management, and Infrastructure as Code (IaC).
- Experience with Prometheus, Grafana, NVIDIA SMI, or DCGM for monitoring and observability.
LLM Deployment and Performance Optimization
- Experience deploying LLM applications on GPU-enabled cloud instances or on-premises GPU servers.
- Familiarity with vLLM, Ollama, Hugging Face TGI, NVIDIA Triton, or equivalent model-serving frameworks.
- Understanding of API-based model serving, GPU resource allocation, and VRAM management.
- Ability to troubleshoot model loading failures, CUDA errors, out-of-memory issues, and inference performance problems.
- Experience optimizing GPU utilization, inference latency, and throughput is desirable.
Experience Requirements
- Minimum 3 years of overall experience in DevOps, SRE, Platform Engineering, or Infrastructure Engineering.
- At least 1–2 years of hands-on experience working with GPU environments.
- Demonstrated involvement in at least one real-world GPU infrastructure setup, GPU cluster deployment, or AI/LLM infrastructure project.
- Ability to explain the GPU hardware, infrastructure architecture, individual responsibilities, deployment steps, and technical issues resolved.
- Production infrastructure support experience is preferred.
Preferred Qualifications
- End-to-end GPU cluster deployment using NVIDIA Base Command Manager.
- Hands-on experience with InfiniBand configuration and troubleshooting.
- Bare-metal GPU server installation and configuration.
- Multi-GPU and multi-node cluster deployment.
- Production LLM serving and GPU performance optimization.
- Experience supporting on-premises, cloud, and hybrid infrastructure environments.
- Experience supporting AI/ML or HPC workloads in production.
Technical Screening Requirements
Candidates should be prepared to discuss the following during the technical interview:
- Describe a real GPU infrastructure or on-premises AI infrastructure project.
- Identify the GPU hardware used, such as NVIDIA A100, H100, H200, L40S, RTX, or equivalent.
- Explain individual responsibilities in GPU setup, cluster configuration, NVIDIA driver installation, CUDA configuration, or LLM deployment.
- Describe a technical issue encountered during deployment and explain the troubleshooting steps taken to resolve it.
- Explain practical experience with Linux, Docker, Kubernetes, NVIDIA BCM, InfiniBand, or GPU monitoring tools.
- Describe an LLM deployment or GPU workload configured, including the environment, tools, and validation process, where applicable.
Ideal Candidate Profile
The ideal candidate should be capable of independently configuring GPU environments, troubleshooting infrastructure and workload issues, automating deployment processes, and supporting reliable AI/ML and HPC operations.
Candidates with strong hands-on experience in GPU server setup, cluster bring-up, NVIDIA drivers, CUDA, and production AI infrastructure will be given preference.
Candidates with only theoretical knowledge of GPUs or general DevOps experience without practical GPU infrastructure exposure will not be suitable for this position.
How to Apply
Interested candidates can share their updated resumes with details of their GPU infrastructure experience, GPU hardware exposure, and relevant AI/LLM deployment projects.
Company: ACEABLE IT LLC
Location: Chennai, India
Pay: ₹1,000,000.00 - ₹1,500,000.00 per year
Work Location: In person
