49 Evaluation Frameworks Jobs

Job Description : The selected candidate will be part of the IWM IT BU, leading Data and AI teams with a strong focus on building a scalable enterprise Data Platform and designing, developing, and deploying AI solutions for IWM business needs. The role involves architecting a unified platform for structured and unstructured data, ensuring efficient data ingestion with minimal redundancy, integrating data virtualization tools such as Denodo or Dremio, and enabling analytics and reporting through APIs, Power BI, real-time streaming, and monitoring dashboards. The candidate will also design and deliver production-grade AI applications using Python and LLM frameworks, including RAG-based systems...

For an international project in Chennai, we are urgently looking for a Full Remote Senior Data Scientist. We are looking for a motivated contractor. Candidates need to be fluent in English. Tasks and responsibilities: Audit existing intranet infrastructure and data systems; Design AI architecture (e.g., local LLM integration, vector database, API design); Integrate existing LLMs suitable for solutions; Develop secure connectors to internal data repositories (SharePoint, document servers, etc.); Create user-facing interfaces or chatbots embedded in the intranet; Implement access control and data governance mechanisms; Validate AI model outputs for accuracy, bias, and relevance; Optimize model...

Job Description We are seeking an experienced Machine Learning Engineer with a strong background in Natural Language Processing (NLP) and building end-to-end ML systems. You should have deep expertise in information retrieval/search, vector databases and LLMOps. In the near term, you will lead production-grade RAG (Retrieval-Augmented Generation) services that power customer support at scale - driving accuracy, latency, and coverage across the entire pipeline, from ingestion and indexing through retrieval, reranking, and LLM orchestration. This role will expand to core AI initiatives for Financial Solutions to build ML/LLM Systems in alignment with our future roadmap. Responsibilities 1.Own ...

About Us: Paytm is India's leading mobile payments and financial services distribution company. Pioneer of the mobile QR payments revolution in India, Paytm builds technologies that help small businesses with payments and commerce. Paytm's mission is to serve half a billion Indians and bring them to the mainstream economy with the help of technology. Paytm has been a pioneer in the merchant space by introducing innovative solutions like QR codes to accept payments and Sound-box to recon payments by voice alerts. We are also distributing loans to these partners via our Paytm for Business App. About the Role: We're looking for a Lead AI engineer with 6-10 years of experience, who can build pro...

Location: Mohali Shift: 09:30 PM - 6:30 AM IST Role Summary You will lead and scale the entire US recruitment function. This senior leadership role requires someone who can operate with full ownership, build high-performing teams, and deliver consistent hiring outcomes across Sales, Leadership, and Senior Management roles. You will refine processes, enforce standards, and run a disciplined, metrics-driven recruitment operation capable of both high-volume and high-quality hiring. Key Responsibilities Talent Acquisition Leadership Own full-cycle recruitment for US-based roles across Sales, Operations, Leadership, and specialized positions Build and execute recruiting strategies that generate s...

Role Overview: As a Senior Training Specialist at Amura Vision, you will be a highly experienced individual contributor responsible for independently managing and executing complex, high-priority learning initiatives that drive organizational performance and employee development. Your expertise in the full training lifecycle, from needs analysis to designing robust Learning Programs and evaluating their impact, will be crucial in achieving our vision of a future with healthier brains in healthier bodies. Key Responsibilities: - Lead the consultation process with senior stakeholders and department heads to accurately identify skill gaps, performance challenges, and strategic learning prioriti...

Hi, We are looking for AI Architect (Enterprise & Generative AI) Exp: 10+ yrs Required Skills & Competencies Core Technical Skills Strong architecture background across cloud, microservices, API ecosystems, and data architecture. Advanced understanding of LLMs, embeddings, vector databases, and evaluation frameworks. Experience with LangChain, LlamaIndex, semantic caching, chunking strategies, and retrieval tuning. Deep knowledge of both traditional ML and modern GenAI approaches. Architecture & Governance Skills TOGAF-certified or strong working knowledge of enterprise architecture practices. Experience in creating Architecture Decision Records (ADRs), system context maps, solution blueprin...

HEAD OF US RECRUITMENT Location: On-site Shift: 09:30 PM - 6:30 AM IST Role Summary You will lead and scale the entire US recruitment function. This senior leadership role requires someone who can operate with full ownership, build high-performing teams, and deliver consistent hiring outcomes across Sales, Leadership, and Senior Management roles. You will refine processes, enforce standards, and run a disciplined, metrics-driven recruitment operation capable of both high-volume and high-quality hiring. Requirements Talent Acquisition Leadership Own full-cycle recruitment for US-based roles across Sales, Operations, Leadership, and specialized positions Build and execute recruiting strategies...

About Highspot Highspot is a software product development company and a recognized global leader in the sales enablement category, leveraging cutting-edge AI and GenAI technologies at the core of its robust Software-as-a-Service (SaaS) platform. Highspot is revolutionizing how millions of individuals work worldwide. Through its AI-powered platform, Highspot drives enterprise transformation to empower sales teams through intelligent content management, training, contextual guidance, customer engagement, meeting intelligence, and actionable analytics. The Highspot platform delivers advanced features tailored to business needs, in a modern design that sales and marketing executives appreciate a...

As a Gen AI Architect, you will design and implement generative AI-based solutions while collaborating with cross-functional teams. You will lead the design, governance, and industrialization of enterprise-grade GenAI platforms and solutions across engineering and manufacturing domains. This role combines deep technical architecture, strategic advisory, and delivery governance, enabling scalable, secure, and business-aligned GenAI transformation for global clients across multiple Business domains. Qualifications Master's or Bachelor's degree in computer science, Data Science, AI, Statistics, Machine Learning, or related fields Strong problem-solving and critical-thinking abilities to analyze...

Role Overview: As the AI Quality & Safety Engineering Lead, your primary responsibility is to ensure that AI systems across the enterprise are safe, fair, reliable, compliant, and observable from prerelease through continuous operations. You will be designing and operating enterprise evaluation frameworks, leading redteaming and guardrail development, encoding policy and regulatory requirements as automated checks, and driving metric reporting and post-incident learning across AI programs. Your role will involve partnering closely with various teams such as Enterprise Architecture, Compliance, Legal/Privacy, Security, Data Governance, and product/engineering teams to enforce quality and safe...

Overview Join the team that powers the most widely used operating system in the world. Windows is evolving into a secure, agent-first platform, and the Windows AI Platform team is at the forefront of this transformation. The team's mission is to lay the foundation for an agent-first Windows experience, driving innovation and reliability for users worldwide. Our Charter: Advancing Windows through Three Pillars Search: Delivering fast, reliable, and relevant system and semantic search capabilities across all Windows surfaces. This enables users to quickly find information and resources, enhancing their productivity and overall experience within the Windows ecosystem. Agentic OS (AgOS): Develop...

Qualification & Eligibility: Bachelor's or higher degree in a quantitative discipline (computer science, statistics, engineering, applied mathematics) Working Experience: Minimum 8+ years of experience Startup experience preferred, Edtech work experience bonus Roles & responsibilities: Lead the roadmap for Generative AI (GenAI) solutions across product and platform teams. Architect and optimise RAG pipelines (retrieval, embeddings, hybrid search, re-ranking, caching, latency-cost tradeoffs). Establish robust evaluation frameworks (automatic + human-in-the-loop) to measure LLM outputs on factuality, reasoning, hallucinations, coverage, and safety. Lead research and applied innovation around f...

At iDreamCareer, we're on a mission to democratize career guidance for millions of young learners across India and beyond. Technology is at the heart of this mission and we're looking for an Engineering Manager who thrives in high-ownership environments, thinks with an enterprising mindset, and gets excited about solving problems that genuinely change lives. This is not just a management role. It's a chance to shape the product, scale the platform, influence the engineering culture, and lead a team that builds with heart and hustle. As a Director of Engineering here, you will: Lead a talented team of engineers while remaining hands-on with architecture and development. Champion the use of AI...

Role Overview: At Capgemini Invent, you will be part of a team that believes in the power of difference to drive change. As a Data Scientist, your role will involve designing, implementing, and optimizing complex AI ecosystems where multiple intelligent agents collaborate to solve sophisticated problems. You will be at the forefront of developing cutting-edge solutions that push the boundaries of artificial intelligence. Key Responsibilities: - Design and develop generative AI-based multi-agent systems that can collaborate, communicate, and coordinate to achieve complex objectives - Architect and implement RAG-based chatbot solutions that effectively leverage knowledge bases and external dat...

Cvent is a leading meetings, events, and hospitality technology provider with more than 4,800 employees and 22,000 customers worldwide, including 53% of the Fortune 500. Founded in 1999, Cvent delivers a comprehensive event marketing and management platform for marketers and event professionals and offers software solutions to hotels, special event venues and destinations to help them grow their group/MICE and corporate travel business. Our technology brings millions of people together at events around the world. In short, we're transforming the meetings and events industry through innovative technology that powers human connection. The DNA of Cvent is our people, and our culture has an emph...

Cvent is a leading meetings, events, and hospitality technology provider with more than 4,800 employees and 22,000 customers worldwide, including 53% of the Fortune 500. Founded in 1999, Cvent delivers a comprehensive event marketing and management platform for marketers and event professionals and offers software solutions to hotels, special event venues and destinations to help them grow their group/MICE and corporate travel business. Our technology brings millions of people together at events around the world. In short, we're transforming the meetings and events industry through innovative technology that powers human connection. The DNA of Cvent is our people, and our culture has an emph...

About Highspot Highspot is a software product development company and a recognized global leader in the sales enablement category, leveraging cutting-edge AI and GenAI technologies at the core of its robust Software-as-a-Service (SaaS) platform. Highspot is revolutionizing how millions of individuals work worldwide. Through its AI-powered platform, Highspot drives enterprise transformation to empower sales teams through intelligent content management, training, contextual guidance, customer engagement, meeting intelligence, and actionable analytics. The Highspot platform delivers advanced features tailored to business needs, in a modern design that sales and marketing executives appreciate a...

As an AI Engineer, you will be responsible for designing, building, and operating agentic AI systems end-to-end, from concept to production. Your main focus will be on multi-agent orchestration, Retrieval-Augmented Generation (RAG), evaluation frameworks, and AI guardrails to ensure the development of safe, reliable, and high-performing systems. Key Responsibilities: - Design, build, and operate agentic AI systems from concept to production. - Work on multi-agent orchestration and Retrieval-Augmented Generation (RAG) to enhance system performance. - Develop evaluation frameworks and AI guardrails to ensure the safety and reliability of systems. Qualifications Required: - Strong background in...

As a potential candidate for the position at the top IT product company, you will be responsible for leading the strategic evolution of India's most sophisticated BFSI AI platform ecosystem. This will involve directing a 40-person engineering and data science team to deliver agentic platform capabilities through cutting-edge agentic tech stack. Your role will encompass agent creation and orchestration, AI-native vertical-focused workflows and journeys, along with scalable AI/ML infrastructure. **Key Responsibilities**: - Define and lead AI platform technology strategy, driving innovation across agentic, low-code, and document science platforms, advanced LLM search, and next-gen financial pro...

Role Overview: You are sought after by Xpergia, a client of Uplers, as an AI/ML Engineer with expertise in conventional machine learning and interest in generative AI. Your primary responsibility will be to develop Sports Companion GPT - Aiko for sports applications, focusing on Cricket and Football. You will collaborate with global teams to create an AI model that provides actionable insights and innovative solutions, contributing to the development of a generative AI evaluation framework. Key Responsibilities: - Design and develop AI models combining conventional ML techniques with generative AI models for Cricket and Football applications. - Fine-tune pre-trained generative AI models usin...

Overview Cvent is a leading meetings, events, and hospitality technology provider with more than 4,800 employees and 22,000 customers worldwide, including 53% of the Fortune 500. Founded in 1999, Cvent delivers a comprehensive event marketing and management platform for marketers and event professionals and offers software solutions to hotels, special event venues and destinations to help them grow their group/MICE and corporate travel business. Our technology brings millions of people together at events around the world. In short, we're transforming the meetings and events industry through innovative technology that powers human connection. The DNA of Cvent is our people, and our culture ha...

As a Compensation & Benefits professional, you will be required to develop, coordinate, and implement effective strategies, procedures, and programs related to compensation and benefits during consulting assignments for ANSR's clients. You will also provide inputs, guidance, and support in setting up and scaling new Global Capability Centers (GCCs). Key Responsibilities: - Work with the People, Organization, and Culture team to deliver high-quality, effective Compensation and Benefits solutions tailored for ANSR's clients with a focus on the following areas: - Aligning with statutory compliance and staying aware of market practices and developments - Conducting Compensation and Benefits mark...

Role Overview: You will be leading the roadmap for Generative AI (GenAI) solutions across product and platform teams. Your responsibilities will include architecting and optimizing RAG pipelines, establishing evaluation frameworks, leading research and applied innovation, mentoring senior scientists/engineers, and partnering with Product, Engineering, and Business stakeholders to deliver AI systems with measurable business impact. Key Responsibilities: - Lead the roadmap for Generative AI (GenAI) solutions - Architect and optimize RAG pipelines - Establish evaluation frameworks to measure LLM outputs - Lead research and applied innovation around foundation model fine-tuning - Mentor senior s...

As a senior-level, applied AI engineer in our team, you will be responsible for building, deploying, and refining consumer-facing AI applications. Your main focus will be on developing conversational AI agents and other generative systems to enhance user interactions with AI features. This role offers you the opportunity to transition from prototyping to building reliable, production-grade solutions. **Key Responsibilities:** - Lead the development and enhancement of the agentic chatbot, including the integration of different chatbot architectures. - Create intuitive and efficient applications that leverage AI interactions to add value and improve user experiences. - Evaluate the AI chatbot ...