Staff AI Scientist

What you ll do

• Dataset Preparation & Evaluation

• Design, prepare, and curate high-quality evaluation datasets with defensible methodology.
• Define criteria for dataset construction, ensuring statistical rigor, reproducibility, and fairness.
• Develop new metrics and evaluation frameworks to measure model performance in nuanced ways.
  

• Model Evaluation & ML Engineering

• Evaluate LLMs and other pre-trained models using carefully chosen datasets and metrics.
• Build scalable pipelines for training, fine-tuning, and benchmarking models.
• Contribute to projects involving fine-tuning, retrieval-augmented generation (RAG), and other adaptation methods.
  

• Cross-Functional Leadership

• Partner with product and engineering to align scientific rigor with business outcomes.
• Define evaluation standards and ML lifecycle practices that raise the bar across the company.
• Mentor scientists and engineers, guiding best practices in experimentation, statistics, and ML development.

Who you are

• Education: Master s degree (PhD preferred) in Computer Science, Statistics, Machine Learning, or a related quantitative field.
• Strong background in mathematical and statistical foundations of machine learning (probability, linear algebra, optimization, experimental design).
• Demonstrated experience in end-to-end ML lifecycle: dataset preparation, model training, evaluation, deployment, and monitoring.
• Proven expertise in evaluation dataset design and metric creation, not just using existing benchmarks but knowing when and how to improve them.
• Experience with LLM evaluation, fine-tuning, and RAG, with the engineering skills to build production-ready pipelines.
• Track record of strategic impact at a staff or principal level setting evaluation and research standards across teams.

Even better if you have

• Publications, open-source contributions, or widely adopted benchmarks in ML/AI.
• Prior experience shaping AI evaluation frameworks at an organizational or industry level.
• Comfort working at the interface of science, engineering, and product.

You will thrive in role if

• You define gold-standard evaluation datasets and metrics that internal teams adopt as a standard.
• You ship datasets and pipelines that are statistically rigorous, reproducible, and scalable.
• You elevate the organization s ML practice by embedding sound statistical and scientific methodology into product and engineering work.
• You are recognized as a thought leader who sets strategy while still contributing technically at depth.