LLM Application / Orchestration Engineer

Company Description

AI Guru builds innovative AI augmentation tools that significantly enhance the capabilities of elite professionals. Our mission is to democratize the use of AI superpowers, ensuring every ambitious professional can benefit from the advanced capabilities traditionally reserved for top consulting firms and Fortune 500 companies. Headquartered by industry veterans who have designed leading AI systems at esteemed organizations like Bloomberg, AWS, and Cerebras, AI Guru's tools are trusted by over 20,000 professionals globally, driving substantial career advancements and operational efficiencies.

Role Description

Design and implement the application layer that connects large language models (LLMs) to real-world data pipelines. You will build and maintain the orchestration logic that retrieves relevant context, feeds it to LLMs, and returns reliable, structured outputs for production systems.

Key Responsibilities

• Architect and maintain the end-to-end LLM orchestration pipeline (retrieval → prompt construction → model call → post-processing).
• Create reusable prompt templates and dynamic context builders for diverse data sources.
• Develop deterministic post-processing and validation layers (schema enforcement, range/regex checks).
• Integrate LLM outputs into backend APIs and user-facing applications.
• Monitor and optimize LLM performance for latency, accuracy, and cost.
• Collaborate with backend, data, and QA teams to improve accuracy and robustness.
• Implement safeguards such as rate limiting, fallback strategies, and prompt versioning.

Required Skills & Experience

• Strong programming skills in

  Python

  or

  TypeScript/Node.js

  for production services.
• Hands-on experience with LLM frameworks (e.g.,

  LangChain, LlamaIndex

  , or similar orchestration tools).
• Expertise in

  prompt engineering

  and structured output handling (e.g., JSON schemas).
• Familiarity with

  vector databases

  (Pinecone, Weaviate, pgvector, etc.) and retrieval strategies.
• Knowledge of CI/CD pipelines, containerization (Docker/Kubernetes), and cloud deployment (AWS/GCP/Azure).
• Strong testing habits for data- and prompt-driven applications.

Nice to Have

• Experience with unstructured data (documents, email, audio, etc.) or information extraction.
• Background in evaluation metrics for retrieval and generation (recall@k, F1, nDCG).
• Understanding of event-driven architectures and message queues (Kafka/SQS).

======

No head hunters please