Urgent Hiring For AI Ops Engineer

Role & responsibilities:

RAG Development & Optimization

• Design and implement Retrieval-Augmented Generation pipelines to ground LLMs in enterprise or domain-specific data.
• Make strategic decisions on chunking strategy, embedding models, and retrieval mechanisms to balance context precision, recall, and latency.
• Work with vector databases (Qdrant, Weaviate, pgvector, Pinecone) and embedding frameworks (OpenAI, Hugging Face, Instructor, etc.).
• Diagnose and iterate on challenges like chunk size trade-offs, retrieval quality, context window limits, and grounding accuracyusing structured evaluation and metrics.

2. Chatbot Quality & Evaluation Frameworks

• Establish comprehensive

  evaluation frameworks

   for LLM applications, combining quantitative (BLEU, ROUGE, response time) and qualitative methods (human evaluation, LLM-as-a-judge, relevance, coherence, user satisfaction).
• Implement continuous monitoring and automated regression testing using tools like

  LangSmith

  ,

  LangFuse

  ,

  Arize

  , or

  custom evaluation harnesses

  .
• Identify and prevent quality degradation, hallucinations, or factual inconsistencies before production release.
• Collaborate with design and product to define

  success metrics

   and

  user feedback loops

   for ongoing improvement.

Minimum Qualifications

• Strong proficiency in

  Python

   (FastAPI, Flask, asyncio) and GCP experience is good to have
• Demonstrated

  hands-on RAG implementation experience

   with specific tools, models, and evaluation metrics.
• Practical knowledge of

  agentic frameworks

   (LangGraph, LangChain) and

  evaluation ecosystems

   (LangFuse, LangSmith).
• Excellent

  communication skills

  , proven ability to

  collaborate cross-functionally

  , and a

  low-ego, ownership-driven

   work style.

Preferred / Good-to-Have Qualifications

• Experience in

  traditional AI/ML workflows

    e.g., model training, feature engineering, and deployment of ML models (scikit-learn, TensorFlow, PyTorch).
• Familiarity with

  retrieval optimization

  ,

  prompt tuning

  , and

  tool-use evaluation

  .
• Background in

  observability and performance profiling

   for large-scale AI systems.
• Understanding of

  security and privacy

   principles for AI systems (PII redaction, authentication/authorization, RBAC)
• Exposure to

  enterprise chatbot systems

  ,

  LLMOps pipelines

  , and

  continuous model evaluation

   in production.