30 Llm Evaluation Jobs

Role Overview: As an Agentic AI using LLMs intern at ViewZen Labs Private Limited, you will be responsible for developing cutting-edge Agentic AI solutions using LangChain and LLMs technology. You will collaborate with the research team to optimize AI models, assist in designing and executing experiments, contribute to the development of new AI applications, stay updated on industry advancements, and support in problem-solving related to AI. Key Responsibilities: - Develop and implement Agentic AI solutions using LangChain and LLMs technology. - Collaborate with the research team to optimize AI models for various projects. - Assist in designing and executing experiments to validate AI soluti...

Role Overview We are looking for AI Evaluation & Data Engineering Specialists to design, curate, and operationalize datasets and evaluation frameworks for AI product performance assessment. This role involves working with large language models (LLMs), human raters, and automation tools to measure model accuracy, correctness, and usability. Key Responsibilities Build and maintain evaluation datasets for AI models across programming languages (Python, Golang, JavaScript, Java). Develop and apply data labeling and scoring guidelines based on Google's evaluation framework. Implement LLM-judge calibration workflows to align automated and human evaluations. Perform error analysis, drift detection,...

As a ML/Dev Ops Engineer at DHI Consultancy, you will be responsible for the following: - Experience: You should have 5 to 10 years of experience in ML Engineering, MLOps, or DevOps roles with a focus on machine learning systems. - Immediate Joiner: You must be able to start immediately or within a very short notice period. - MLOps Tools: Expert-level proficiency with mlFlow, Seldon, and Kubeflow is required. - Data & Feature Engineering: Strong practical experience with Databricks and Tecton (or similar feature stores) is essential. - Cloud & Infrastructure: Deep knowledge of AWS services for computing, storage, networking, and security is necessary. - CI/CD & Automation: Proven ability to ...

GlobalLogic is a digital engineering company, a subsidiary of the Hitachi Group, that specializes in helping large companies design, build, and improve digital products, platforms, and services. Headquartered in Silicon Valley, it operates engineering centers and design studios globally, serving various industries like automotive, financial services, healthcare, and technology. Founded in 2000, the company was acquired by Hitachi in 2021 and combines its digital engineering expertise with Hitachi's Lumada solutions. QA Leader to drive quality across complex IoT and GenAI programs. The candidate should be capable of leading difficult conversations with Japanese customers while maintaining str...

About Company : They balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society. About Client: Our client is a global digital solutions and technology consulting company headquartered in Mumbai, India. The company generates annual revenue of over $4.29 billion (?35,517 crore), reflecting a 4.4% year-over-year growth in USD terms. It has a workforce of around 86,000 professionals operating in more than 40 countries and se...

You will be joining a global digital solutions and technology consulting company headquartered in Mumbai, India, with a strong ethical reputation and innovative culture. With an annual revenue exceeding $4.29 billion and a workforce of 86,000 professionals in over 40 countries, the company serves a diverse global client base across various industries. As an Automation Tester (Ai Testing_LLM+Python), you will play a crucial role in evaluating and optimizing Large Language Model (LLM) responses using frameworks like Ragas, LangSmith, and Python-based tools. Your expertise will contribute to enhancing model accuracy, reliability, and performance, ensuring high-quality GenAI solutions. **Key Res...

You will be responsible for building production-grade Java services and AI Agents that power SAP Sales Cloud V2 capabilities, including Visit & Activity Management, Digital Selling/Call Lists, and Dynamic & Static Route Planning. This will involve working across Kyma/Kubernetes, Kafka, Spring Boot, and modern GenAI stacks such as Spring AI, MCP, vector search, and LLM evaluation. Your work will include ensuring enterprise-grade security, multitenancy, and observability for these services. Key Responsibilities: - Build production-grade Java services and AI Agents for SAP Sales Cloud V2 capabilities - Work with technologies like Kyma/Kubernetes, Kafka, Spring Boot, and modern GenAI stacks - En...

As the Lead AI/ML Architect at this fast-growing health-tech company, you will play a crucial role in designing and building next-gen Agentic AI systems to revolutionize primary care. Your responsibilities will include: - Leading the architecture, design, and implementation of LLM-based and agentic AI systems for clinical and operational use cases. - Overseeing the development of multi-agent orchestration frameworks using tools like LangGraph, CrewAI, or Semantic Kernel. - Building scalable RAG pipelines and retrieval systems using vector databases like Pinecone, FAISS, Weaviate, Vertex AI Matching Engine. - Guiding engineers on prompt design, model evaluation, multi-step orchestration, and ...

Role Overview: You will be responsible for leveraging your strong technical skills in Python, SQL, machine learning concepts, and data visualization tools to work on data projects. Your role will involve developing data pipelines, optimizing database queries, and ensuring data quality and validation. Additionally, you will be required to have expertise in AI and prompt engineering, including prompt design, application development, and AI model deployment. Key Responsibilities: - Strong proficiency in Python with Pandas, SQL expertise, and version control systems (Git) - Experience with data pipeline development, ETL processes, and data warehouse design - Knowledge of statistical methods, mac...

HackerRank helps thousands of companies like OpenAI, NVIDIA, Amazon hire developers based on their skills vs pedigree, and also nurtures a community of millions of developers to upskill themselves to become the next-gen developers. The people at HackerRank care deeply about their work and have an extremely intense work ethic. In many companies, speed & quality is a tradeoff. At HackerRank, it's not -- we expect you to ship in about half the time that most competent people think it's possible while maintaining a standard of quality you'd proudly sign your name on. The only way to make this happen is if you truly love your craft and are deeply committed to growth. About The Role We are seeking...

About this role: Wells Fargo is seeking a Quantitative Analytics Specialist. In this role, you will: Develop, implement, and calibrate various analytical models Perform highly complex activities related to financial products, business analysis and modeling Perform basic statistical and mathematical models using Python, R, SAS, C++ and SQL Perform analytical support and provide insights regarding a wide array of business initiatives Provide solutions to business needs and analyze workflow processes to make recommendations for process improvement in risk management Collaborate and consult with peers, colleagues, managers, and regulators to resolve issues and achieve goals Required Qualificatio...

Opportunity to work with our client who is building the AI foundation for value-based healthcare we're hiring an AI/LLM Technical Lead to architect next-gen Agentic AI systems transforming how healthcare is delivered. Responsibilities: ? Lead architecture, design, and implementation of LLM-based and agentic AI systems for clinical and operational use cases. ? Oversee the development of multi-agent orchestration frameworks (reasoning, planning, and task execution) using tools such as LangGraph, CrewAI, or Semantic Kernel . ? Build scalable RAG pipelines and retrieval systems using vector databases (Pinecone, FAISS, Weaviate, Vertex AI Matching Engine). ? Guide engineers on prompt design, mode...

You are a driven and innovative individual looking to make a mark in the world of artificial intelligence. ViewZen Labs Private Limited is seeking a talented Agentic AI using LLMs intern with expertise in Python, LangChain, and LLM evaluation to join their dynamic team. As an Agentic AI using LLMs intern at ViewZen Labs Private Limited, you will have the opportunity to: - Develop and implement cutting-edge Agentic AI solutions using LangChain and LLMs technology. - Collaborate with the research team to optimize AI models for various projects. - Assist in the design and execution of experiments to validate AI solutions. - Contribute to the development of new AI applications and features. - St...

We are looking for an AI Lead to drive innovation, strategy, and delivery of cutting-edge AI solutions across multiple domains. The ideal candidate will combine hands-on expertise in AI/ML/GenAI with strong leadership skills to guide teams and deliver impactful business outcomes. Key Responsibilities Lead the design, development, and deployment of AI/ML/Generative AI solutions across diverse business domains. Provide technical leadership on LLMs, Generative AI, NLP, Deep Learning, and Machine Learning projects. Oversee end-to-end AI lifecycle from data collection, preprocessing, model design, fine-tuning, evaluation, to deployment and monitoring. Drive initiatives on LLM fine-tuning, prompt ...

As a ML/Dev Ops Engineer at DHI Consultancy, you will be responsible for: - Having 5 to 10 years of experience in ML Engineering, MLOps, or DevOps roles with a focus on machine learning systems. - Being an immediate joiner, able to start immediately or within a very short notice period. - Demonstrating expert-level proficiency with mlFlow, Seldon, and Kubeflow MLOps tools. - Possessing strong practical experience with Databricks and Tecton (or similar feature stores) for Data & Feature Engineering. - Having deep knowledge of AWS services for computing, storage, networking, and security for Cloud & Infrastructure. - Showing proven ability to build and manage automated pipelines using Jenkins ...

Description and Requirements At BMC trust is not just a word - it's a way of life! Hybrid Description and Requirements At BMC trust is not just a word - it's a way of life! We are an award-winning, equal opportunity, culturally diverse, fun place to be. Giving back to the community drives us to be better every single day. Our work environment allows you to balance your priorities, because we know you will bring your best every day. We will champion your wins and shout them from the rooftops. Your peers will inspire, drive, support you, and make you laugh out loud!? We help our customers free up time and space to become an Autonomous Digital Enterprise that conquers the opportunities ahead - ...

We are looking for an AI Lead to drive innovation, strategy, and delivery of cutting-edge AI solutions across multiple domains. The ideal candidate will combine hands-on expertise in AI/ML/GenAI with strong leadership skills to guide teams and deliver impactful business outcomes. Key Responsibilities Lead the design, development, and deployment of AI/ML/Generative AI solutions across diverse business domains. Provide technical leadership on LLMs, Generative AI, NLP, Deep Learning, and Machine Learning projects. Oversee end-to-end AI lifecycle from data collection, preprocessing, model design, fine-tuning, evaluation, to deployment and monitoring. Drive initiatives on LLM fine-tuning, prompt ...

It's fun to work in a company where people truly BELIEVE in what they are doing! We're committed to bringing passion and customer focus to the business. Role Overview We're building a next-gen LLMOps team at Fractal to industrialize GenAI implementation and shape the future of GenAI engineering. This is a hands-on technical leadership role for AI engineers with strong ML and DevOps skills ideal for those who love building scalable systems from the ground up. You will be designing, deploying, and scaling GenAI and Agentic AI applications with robust lifecycle automation and observability. Required Qualifications 10 - 14 years of experience in working on ML projects that includes product build...

About the job: Humanity Founders is hiring a Senior AI Engineer to build and ship AI systems that create real, positive social impact. In this full-time, remote role, you'll own the end-to-end lifecycle from problem framing and data pipelines to model training/finetuning, LLM ops, evaluation, and production deployment. You'll work closely with product, design, and domain experts, leading technical decisions and mentoring teammates while holding a high bar for quality, safety, and velocity. You'll architect reliable ML/LLM services, design robust evaluation and observability, and turn research into user-facing features. This is a hands-on role for an engineer who thrives in a fast, product-ce...

About the job: As an AI Engineer at Humanity Founders, you will play a crucial role in developing cutting-edge AI solutions that have a positive impact on society. You will work closely with a talented team of engineers and data scientists to build and deploy innovative AI applications. Key responsibilities: 1. Develop and implement Machine Learning Operations (MLOps) strategies to streamline the development and deployment of AI models. 2. Utilize your expertise in Machine Learning, Artificial Intelligence, Neural Networks, LLM Ops, LLM evaluation, Training, and Development to enhance our AI capabilities. 3. Lead and participate in the prompt engineering of AI solutions using technologies su...

As a Senior AI/ML Engineer with 8 years of experience based in Bangalore, you will be responsible for leveraging your expertise in Prompt Engineering, Large Language Models (LLMs), and Agentic AI frameworks. Your primary focus will be on designing, implementing, and optimizing AI-driven solutions that capitalize on the latest advancements in generative AI. Your key responsibilities will include developing prompt engineering strategies to enhance AI model performance, fine-tuning LLM hyperparameters like temperature and top_p, building and deploying solutions using Agentic AI frameworks such as LangChain, LangGraph, and Crew AI. You will also be involved in implementing Retrieval-Augmented Ge...

You have a great opportunity with Infosys for the position of Python AI/ML with GEN AI. This is a contract role based in either Chennai or Pune (Onsite). You should have a minimum of 4 years of experience in AI/ML, specifically in GenAI, along with a total of 5+ years of industry experience. Your responsibilities will include demonstrating proficiency in Python, ML, NLP, and text data analysis. You should have hands-on experience in API creation, RAG, Vector databases, LLM prompt engineering, and LLM evaluation. It is important to be familiar with MLOps tools and practices, such as monitoring and evaluation tools like LangSmith, LLM deployment libraries like LiteLLM and LLM Guardrails, CI/CD...

About the job: Are you passionate about the future of AI and ML? We're looking for a developer who doesn't just write code, but co-creates with it. If you see tools like Chat GPT, GitHub Copilot, and Claude as your coding partners and not just a means to an end, then you've found the right place. We believe that great software is built on creative problem-solving and solid logic, not just language-specific syntax. You should be comfortable with any coding language and have a natural curiosity to find smarter, faster ways to solve problems. Have you used AI to debug tricky code or convert a project between languages? If so, you're the kind of person we want in our team. Key Responsibilities: ...

We are urgently looking for a Senior AI/ML Engineer (C2H) based in Bangalore for on-site work. 7+years Key Requirements: - Proficiency in Prompt Engineering and its strategies - Understanding of LLM functionality and its hyperparameters (temperature, top_p) - Experience with Agentic AI frameworks such as Langchain, Lang graph, and Crew AI - Familiarity with RAG - Skilled in LLM evaluation and observability - Strong command of Python - Ability to deploy on any one of the cloud services If you meet these requirements, we would like to hear from you. [HIDDEN TEXT] hashtag#immediatejoiners hashtag#AI hashtag#PYTHON hashtag#Machinelearning hashtag#banglore Show more Show less

About the job: As a Full Stack GenAI Developer at MeetMinutes, you will be responsible for creating cutting-edge AI solutions using Python, Generative AI Development, LangChain, LLM evaluation, LLMOps, JavaScript, React, Amazon Web Services (AWS), Google Cloud Platforms (GCP), Docker, Machine Learning, Natural Language Processing (NLP), PostgreSQL, REST API, FastAPI, GitHub, System Design, and Prompt Engineering. Key responsibilities: 1. Developing and implementing AI algorithms and models to enhance the functionality of our platform. 2. Integrating AI technologies and features into our existing systems to improve user experience. 3. Collaborating with the engineering team to optimize system...