Senior Data and AI Analyst

• Design and maintain comprehensive analytics frameworks for business processes and financial workflows
• Develop dashboards and reports that provide real-time insights into key business metrics
• Analyze user interaction patterns with AI features to improve adoption and effectiveness
• Create and maintain ETL pipelines (if necessary) to ensure data quality and accessibility
• Generate actionable insights from complex financial datasets to guide product strategy
  

• Develop and implement frameworks to evaluate the performance of LLM-based systems and AI agent.s
• Analyze AI model outputs, response quality, and automation effectiveness
• Track and report on key AI performance metrics such as accuracy, latency, and automation rates
• Collaborate with ML engineers to identify areas for model improvements based on data insights
• Create dashboards for monitoring AI system performance and reliability
• Conduct A/B testing of different AI configurations and prompt strategies
  

• Provide data-driven recommendations for improving both AI capabilities and business processes
• Partner with product teams to define and track success metrics for AI features
• Collaborate with customer success teams to analyze user feedback and AI system effectiveness
• Build and maintain documentation for AI performance metrics and analytics processes
  

Qualifications

• 5+ years of experience in data analysis, with recent exposure to AI/ML systems
• Strong proficiency in SQL and Python for data manipulation and analysis
• Experience analyzing AI/ML system performance and metrics
• Expert-level experience with business intelligence tools (e.g., Tableau, Power BI, Looker)
• Experience working with large-scale financial datasets and ETL processes
• Strong statistical analysis skills and experience with A/B testing
• Experience with cloud-based data warehouses (preferably Snowflake)
• Bachelor’s degree in Statistics, Mathematics, Computer Science, or related field
  

Preferred Qualifications

• Experience analyzing LLM-based systems and generative AI applications
• Knowledge of NLP metrics and evaluation frameworks
• Experience with financial systems and processes (AP, AR, Vendor Management)
• Familiarity with prompt engineering and LLM performance optimization
• Experience with real-time analytics and streaming data
• Knowledge of data privacy and compliance requirements in finance
• Master’s degree in related field
  

• Python (Pandas, NumPy, Matplotlib, Seaborn, Plotly, Dash)
• SQL (expert-level analytics)
• Tableau/Power BI/QuickSight
• Excel (advanced)
  

• LLM evaluation metrics
• AI monitoring frameworks
• Prompt effectiveness analysis
  

• Snowflake or similar data warehouses
• AWS/Azure data services
• Experience with ETL tools and frameworks
• Data modeling
  

• Statistical analysis
• A/B testing
• Hypothesis testing
• Experimental design