ML Inference & Optimization Engineer

ML Inference & Optimization Engineer Location: Mumbai, Experience: 2–4 years You will be responsible for deploying and scaling domain and task-specific LLMs and deep learning models for real-time and batch inference. You'll work on quantization, model optimizations, runtime tuning, and performance-critical serving. What You'll Do Integrate models into containerized services and APIs, and build high-performance inference pipelines optimized for latency, concurrency, and cost Deploy and optimize LLMs using vLLM, TGI, SGLang, Triton, TensorRT etc. Implement model quantization, speculative decoding, KV cache optimization, dynamic batching etc. Benchmark model throughput and latency across cloud VM configurations Debug performance bottlenecks: VRAM usage, token sampling speed, latency, instability Collaborate with infra team for scaling and observability Monitor and troubleshoot inference performance, ensuring system reliability and efficiency Stay abreast of advancements in model inference technologies and best practices You Bring 3+ years of experience in deploying and optimizing machine learning models in production, with 1+ years of experience in deploying deep learning models Experience deploying async inference APIs (FastAPI, gRPC, Ray Serve etc.) Understanding of PyTorch internals and inference-time optimization Familiarity with LLM runtimes: vLLM, TGI, TensorRT-LLM, ONNX Runtime etc. Familiarity with GPU profiling tools (nsight, nvtop), model quantization pipelines Bonus: prior work on ElasticSearch, distributed KV cache, or custom tokenizers Bachelor's degree in Computer Science, Engineering, or related field Show more Show less