Agentic System Design:
Design, develop, and deploy
autonomous AI agents
capable of complex, goal-oriented reasoning, planning, and task execution using modern agentic frameworks.
RAG System Development:
Build and optimize robust
Retrieval-Augmented Generation (RAG)
pipelines to ground LLMs in proprietary data, ensuring factual accuracy and data security.
Tool Integration & Function Calling:
Equip AI agents with the ability to use external APIs, databases, and custom tools to perform actions in the real world.
Orchestration Frameworks:
Master and implement core logic using orchestration tools like
LangChain, LlamaIndex, LangGraph, or CrewAI
to manage conversation state and agent workflows.
Prompt Engineering & Alignment:
Develop systematic Prompt Engineering strategies to maximize agent reliability and output quality.
Production Deployment:
Develop secure, low-latency API services (using
Python/FastAPI
) to serve LLM applications, ensuring high availability and scalability.
Evaluation & Quality Assurance (Evals):

Design and implement continuous evaluation frameworks
to measure performance against business metrics. This includes developing
ground truth datasets
, implementing
LLM-as-a-Judge
scoring, and monitoring for
hallucinations, factual correctness, and prompt injection
.

Required Skills and Qualifications

Programming:

3+ years
of professional software development experience, with expert proficiency in
Python
.
LLM Application Experience:
Direct experience building and deploying production-level applications using major LLM APIs or open-source models.
Core RAG Expertise:
Deep practical knowledge of designing, implementing, and optimizing a RAG system and proficiency with
Vector Databases
(e.g., Pinecone, Chroma, Milvus).
Agent Orchestration:
Proven experience with at least one major agent orchestration library (
LangChain, LlamaIndex, or similar state machine/graph-based frameworks
).
Evaluation Tools:
Hands-on experience with LLM evaluation frameworks like
RAGAS, DeepEval, or LangSmith
to create automated performance benchmarks.
Back-end Engineering:
Strong experience developing and maintaining RESTful APIs and integrating with various data sources (SQL/NoSQL).
DevOps Fundamentals:
Working knowledge of
Docker, CI/CD pipeline
and cloud infrastructure for deployment.

More Jobs at Codevally

IT Marketer (On-Site | B2B Marketing)

ahmedabad, gujarat, india

5.0 - 5.0 yrs

Salary: Not disclosed

LLM Application Developer

ahmedabad, gujarat, india

3.0 - 3.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.