Data Scientist

Overview

Epicor Prism is the foundation for AI development across the company, providing the common tooling, orchestration frameworks, telemetry, and prompt infrastructure that powers dozens of agents in areas like compliance, automation, and analytics.

As a Data Scientist on the Prism team , your work will directly impact how LLM applications are built, tested and deployed at Epicor. You’ll collaborate with platform engineers, senior AI scientists, and product teams to improve evaluation quality, prompt test harnesses, experiment tracking, and context management tooling —ensuring every agent built on Prism is observable, testable, and reproducible.

This role is ideal for a data scientist who wants to go beyond applying models and instead build the internal foundations that make applied AI scalable and safe.

What You’ll Do

• Design and maintain evaluation pipelines for prompt/agent reliability, accuracy, and latency
• Build prompt experimentation tools (e.g., versioning, benchmarking, result comparison) used by internal teams
• Help improve context management strategies , chunking, summarization, and vector-based retrieval
• Contribute to dashboards and telemetry pipelines to track agent health, drift, and usage patterns
• Work closely with platform engineers to validate experiment outputs and improve system behavior
• Collaborate with data scientists from other teams to ensure Prism meets their experimentation and deployment needs
• Document and advocate for best practices in LLM evaluation, experiment design, and prompt testing

Required Qualifications

• 2–3 years of experience in data science or applied ML roles, ideally involving AI systems or infrastructure
• Strong Python skills, including experience with APIs, data processing, and testing frameworks
• Hands-on experience designing and running structured experiments, including metric design and result interpretation
• Familiarity with LLM fundamentals, prompt engineering concepts and evaluation techniques
• Comfortable working in a collaborative, platform-style team that builds tools for other teams to use
• Solid communication skills—able to document findings clearly and explain tradeoffs

Nice to Have

• Experience with LLM evaluation libraries (e.g., TruLens, RAGAS), test harnesses, or CI for prompts
• Familiarity with vector stores and retrieval (e.g., FAISS, Weaviate, Qdrant)
• Exposure to prompt versioning, agent behavior logging, or result diffing tools
• Experience with vector databases, retrieval pipelines, or context management approaches
• Background in developer tooling, infrastructure teams, or MLOps
• Background in ERP, compliance, or supply chain
• Exposure to .NET/C# environments and lightweight deployment workflows (containers, APIs, CI/CD)

Success Metrics

• Improved accuracy, consistency, and observability across agents built on Prism
• Internal teams adopt and reuse your evaluation tools, experiment templates, or context modules
• Reduced incident rates or regressions in deployed agents through better pre-deployment testing
• Positive internal feedback from teams using Prism’s evaluation and experimentation capabilities

This Role Is Ideal For…

A data scientist who wants to build AI infrastructure, not just models . Someone who values repeatability, clarity, and experimentation rigor—and wants to support dozens of teams building LLM agents by giving them world-class tools and systems.

#LI-SM1 #LI-HYBRID

Whatever your career journey, we’ll help you find the right path. Through our training courses, mentorship, and continuous support, you’ll get everything you need to thrive. At Epicor, your success is our success. And that success really matters, because we’re the essential partners for the world’s most essential businesses—the hardworking companies who make, move, and sell the things the world needs.

Recruiter: