Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in kolkata
>
Mindrift
>
Freelance AI Evaluation Scenario Writer

Freelance AI Evaluation Scenario Writer

Name: Jobpe
Address: T-Hub, Plot No 1/C, Sy No 83/1, Raidurgam panmaktha, Knowledge City Rd, Hyderabad, Telangana, 500081, IN
Telephone: +91-83339-09630
Price range: Free

Mindrift

3 - 7 years

8 - 11 Lacs

kolkata mumbai new delhi bengaluru

Posted:1 day ago| Platform:

Apply

Skills Required

igcse bed teaching english mathematics myp teaching training coordination curriculum development teaching mathematics content development economics ibdp physics teaching biology isc ib pgt icse cie communication skills

Work Mode

Work from Office

Job Type

Full Time

Job Description

About the Role

We re looking for someone who can design

realistic and structured evaluation scenarios

for LLM-based agents. You ll create test cases that simulate human-performed tasks and define gold-standard behavior to compare agent actions against. You ll work to ensure each scenario is clearly defined, well-scored, and easy to execute and reuse. You ll need a sharp analytical mindset, attention to detail, and an interest in how AI agents make decisions.

Although every project is unique, you might typically:

Create structured test cases that simulate complex human workflows.
Define gold-standard behavior and scoring logic to evaluate agent actions.
Analyze agent logs, failure modes, and decision paths.
Work with code repositories and test frameworks to validate your scenarios.
Iterate on prompts, instructions, and test cases to improve clarity and difficulty.
Ensure that scenarios are production-ready, easy to run, and reusable.

How to get started

Simply apply to this post, qualify, and get the chance to contribute to projects aligned with your skills, on your own schedule. From creating training prompts to refining model responses, you ll help shape the future of AI while ensuring technology benefits everyone.

Bachelors and/or Master s Degree in Computer Science, Software Engineering, Data Science / Data Analytics, Artificial Intelligence / Machine Learning, Computational Linguistics / Natural Language Processing (NLP), Information Systems or other related fields.
Background in QA, software testing, data analysis, or NLP annotation.
Good understanding of
test design principles
(e.g., reproducibility, coverage, edge cases).
Strong written communication skills in English.
Comfortable with structured formats like JSON/YAML for scenario description.
Can define expected agent behaviors (gold paths) and scoring logic.
Basic experience with Python and JS.
Curious and open to working with AI-generated content, agent logs, and prompt-based behavior.

Nice to Have

Experience in writing manual or automated test cases.
Familiarity with LLM capabilities and typical failure modes.
Understanding of scoring metrics (precision, recall, coverage, reward functions).

Contribute on your own schedule, from anywhere in the world. This opportunity allows you to:

Get paid for your expertise, with
rates that can go up to $12/hour
depending on your skills, experience, and project needs.
Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments.
Participate in an advanced AI project and gain valuable experience to enhance your portfolio.
Influence how future AI models understand and communicate in your field of expertise.

More Jobs at Mindrift

Freelance AI Agent Assistant

Mumbai Metropolitan Region

Experience: Not specified

Salary: Not disclosed

Freelance Consulting Expert - AI Trainer (Talent pool)

Mumbai Metropolitan Region

3.0 - 3.0 yrs

Salary: Not disclosed

Freelance AI Agent Assistant

New Delhi, Delhi, India

Experience: Not specified

Salary: Not disclosed

Freelance Consulting Expert - AI Trainer (Talent pool)

New Delhi, Delhi, India

3.0 - 3.0 yrs

Salary: Not disclosed

Freelance Consulting Expert - AI Trainer (Talent pool)

India

3.0 - 3.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now

Mindrift

Login to

Please Verify Your Phone or Email

Confirm Action

Freelance AI Evaluation Scenario Writer