AI Systems Engineer

SDE 2 / SDE 3 – AI Infrastructure & LLM Systems Engineer

Location:

Experience:

Compensation:

Bonus:

About The Company

production-grade AI systems for BFSI enterprises

run our own LLMs on GPUs

Role Summary (What We’re Really Hiring For)

strong backend / systems engineer

• Deploy AI models on GPUs
• Expose them via APIs
• Scale inference under high parallel load using async systems and queues
  

not

Core Responsibilities

• Deploy and operate LLMs on GPU infrastructure (cloud or on-prem).
• Run inference servers such as vLLM / TGI / SGLang / Triton or equivalents.
• Build FastAPI / gRPC APIs on top of AI models.
• Design async, queue-based execution for AI workflows (fan-out, retries, backpressure).
• Plan and reason about capacity & scaling:
• GPU count vs RPS
• batching vs latency
• cost vs throughput
• Add observability around latency, GPU usage, queue depth, failures.
• Work closely with AI researchers to productionize models safely.
  

Must-Have Skills

• Strong backend engineering fundamentals (distributed systems, async workflows).
• Hands-on experience running GPU workloads in production.
• Proficiency in Python (Golang acceptable).
• Experience with Docker + Kubernetes (or equivalent).
• Practical knowledge of queues / workers (Redis, Kafka, SQS, Celery, Temporal, etc.).
• Ability to reason quantitatively about performance, reliability, and cost.
  

Strong Signals (Recruiter Screening Clues)

Look For Candidates Who Have

• Personally deployed models on GPUs
• Debugged GPU memory / latency / throughput issues
• Scaled compute-heavy backends under load
• Designed async systems instead of blocking APIs
  

Nice to Have

• Familiarity with LangChain / LlamaIndex (as infra layers, not just usage).
• Experience with vector DBs (Qdrant, Pinecone, Weaviate).
• Prior work on multi-tenant enterprise systems.
  

Not a Fit If

• Only experience is calling OpenAI / Anthropic APIs.
• Primarily a prompt engineer or frontend-focused AI dev.
• No hands-on ownership of infra, scaling, or production reliability.