LLM Ops Engineer - Serverless & CI/CD (AWS)

The Role

The Responsibilities

• Design, implement, and maintain

  CI/CD pipelines

  for

  Agentic AI applications

  using

  Terraform

  ,

  AWS CodePipeline

  ,

  CodeBuild

  , and related tools.
• Automate deployment of multi-agent systems and associated tooling, ensuring version control, rollback strategies, and consistent environment parity across dev/test/prod.

Agent Development & Debugging

• Collaborate with ML/NLP engineers to develop and deploy

  modular, tool-integrated AI agents

  in production.
• Lead the effort to create

  debuggable agent architectures

  , with structured logging, standardized agent behaviors, and feedback integration loops.
• Build agent lifecycle management tools that support

  quick iteration, rollback, and debugging

  of faulty behaviors.

Monitoring, Tracing & Reliability

• Implement

  end-to-end observability

  for agents and tools, including

  runtime performance metrics

  ,

  tool invocation traces

  , and

  latency/accuracy tracking

  .
• Design dashboards and alerting mechanisms to capture

  agent failures, degraded performance, and tool bottlenecks

  in real-time.
• Build lightweight tracing systems that help

  visualize agent workflows

  and simplify root cause analysis.

Cost Optimization & Usage Analysis

• Monitor and manage

  cost metrics

  associated with agentic operations including

  API call usage

  ,

  toolchain overhead

  , and

  model inference costs

  .
• Set up proactive

  alerts for usage anomalies

  , implement

  cost dashboards

  , and propose strategies for reducing operational expenses without compromising performance.

Collaboration & Continuous Improvement

• Work closely with product, backend, and AI teams to evolve the

  agentic infrastructure design

  and

  tool orchestration workflows

  .
• Drive the adoption of

  best practices for Agentic AI DevOps

  , including retraining automation, secure deployments, and compliance in cloud-hosted environments.
• Participate in design reviews, postmortems, and architectural roadmap planning to continuously improve reliability and scalability.

• 2+ years

  of experience in DevOps, MLOps, or Cloud Infrastructure with exposure to

  AI/ML systems

  .

• Deep expertise in AWS serverless architecture

  , including hands-on experienc