Industry & Sector:
A fast-growing EdTech and AI-powered online learning platform operating in the education technology and generative AI sector. We build intelligent learning experiences and training data pipelines that improve LLM-based tutoring, automated assessment, and personalized content delivery.
Role Title (standardized):
AI Trainer
Location & Work Type:
India — Remote
About The Opportunity
We are seeking an experienced AI Trainer to join a distributed team focused on creating high-quality training datasets and evaluation workflows for large language models and conversational AI. This role combines hands-on data labeling, annotation guideline design, model evaluation, and continuous improvement to ensure reliable, safe, and pedagogically-sound AI behavior.Role & Responsibilities
- Design, execute, and manage high-fidelity annotation tasks and labeling workflows for text, conversation, and feedback data to support LLM training and evaluation.
- Create and maintain clear annotation guidelines, edge-case examples, and inter-annotator reconciliation processes to ensure label consistency and high dataset quality.
- Perform model-in-the-loop annotation and evaluation: generate prompts, review model outputs, score responses, and provide corrective labels to inform iterative training.
- Conduct quality assurance and statistical validation of labelled datasets; identify systemic errors and implement remediation strategies.
- Collaborate with ML engineers and product managers to translate learning objectives into measurable annotation schemas and evaluation metrics.
- Train and coordinate distributed annotator teams or crowd workers, provide feedback, and track productivity and accuracy KPIs.
Skills & Qualifications
Must-Have
- Data annotation
- Labelbox
- Scale AI
- Natural Language Processing
- Prompt engineering
- Python scripting
Preferred
- LLM evaluation frameworks (e.g., Evals)
- Crowd-platforms (Appen, Amazon Mechanical Turk)
- Multilingual annotation and linguistic analysis
Benefits & Culture Highlights
- Fully remote, flexible work environment with asynchronous collaboration across India.
- Opportunity to shape annotation standards that directly influence production LLM behavior in education products.
- Hands-on learning culture with cross-functional exposure to ML engineering, product, and pedagogical teams.
We welcome candidates who combine domain rigor with curiosity about language models and pedagogy. If you take pride in creating clean, defensible datasets and enjoy translating learning goals into measurable annotation tasks, this role offers a high-impact path to influence AI-driven education at scale.
Skills: ml,python scripting,natural language processing,ai