Data Analyst (Remote)

Job Title:

Location:

About the Role

We are looking for a data-driven professional to join our analytics and quality team. In this role, you will analyze the performance of AI agents across finance-sector tasks to uncover root causes of failure and drive improvements in design, evaluation, and benchmarking frameworks.

Key Responsibilities

• Conduct statistical failure analysis to identify recurring patterns in AI agent performance across prompts, rubrics, templates, and file types
• Investigate root causes behind performance issues in task execution, framework design, or data complexity
• Perform multi-dimensional analysis across finance sub-domains, task categories, and input types
• Develop dashboards and reports highlighting performance trends, failure clusters, and improvement opportunities
• Recommend enhancements to task design, quality rubrics, and evaluation metrics based on data findings

Required Qualifications

• Strong understanding of statistical analysis, hypothesis testing, and pattern recognition
• Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analytics
• Hands-on experience in exploratory data analysis (EDA) and turning data into actionable insights
• Familiarity with AI/ML evaluation techniques and quality assessment metrics
• Proficiency in Excel, SQL, and visualization tools such as Tableau or Looker

Preferred Qualifications

• Prior experience in AI/ML model evaluation, data labeling quality, or automation analysis
• Knowledge of finance-sector concepts or willingness to learn finance domain structures
• Experience with multidimensional failure analysis or benchmarking studies
• Understanding of large-scale evaluation frameworks or test dataset design

Equal Opportunity Statement

We welcome applicants from all backgrounds. Reasonable accommodations are available for qualified individuals with disabilities.