Key Responsibilities AI System Design and Development
- Build and maintain agent workflows, pipelines, and reusable skills that carry work from specification through implementation and review.
- Develop prompt architectures, system instructions, and context assembly strategies, and version them as engineered artifacts under source control rather than ad hoc text.
- Implement retrieval augmented generation pipelines, including document ingestion, chunking, embedding, indexing, and retrieval quality tuning.
- Integrate model providers and tool interfaces, including Model Context Protocol servers and connectors to platforms such as Atlassian, source control, and client systems.
- Build the supporting application layer around AI capability: APIs, background processing, state management, and user facing interaction patterns.
Evaluation, Quality, and Guardrails
- Design and maintain evaluation suites that measure accuracy, groundedness, regression, latency, and cost against defined benchmarks.
- Establish guardrails for input validation, output constraint, prompt injection resistance, and safe failure behavior.
- Define and implement human in the loop review points where autonomous output is not acceptable on its own.
- Instrument systems with tracing, logging, and monitoring sufficient to reconstruct why a given output was produced.
- Track and manage token consumption, model selection, and unit economics per workload.
Compliance and Data Protection
- Build within HIPAA boundaries, including appropriate handling of protected health information, data minimization, and awareness of which workloads may reach which model endpoints.
- Document data flows, model usage, and retention behavior to support security reviews, client due diligence, and audit requests.
- Apply access control, secrets management, and environment separation consistent with Kairos engineering standards.
Delivery and Collaboration
- Work inside an Agile delivery model alongside developers, architects, business analysts, and QA.
- Translate business and clinical workflow requirements into system behavior and push back where a requirement asks a model to do something it cannot do reliably.
- Produce clear technical documentation in Confluence covering architecture, decisions, and operating procedures.
- Participate in code review, contribute to internal engineering standards, and help raise AI fluency across the delivery organization.
Required Qualifications
- 5+ or more years of professional software engineering experience, with at least 3 year building AI or language model powered features that reached production.
- Strong programming proficiency in Python and or TypeScript / JavaScript.
- Demonstrated experience with language model APIs, prompt engineering, and tool or function calling patterns.
- Working knowledge of retrieval architectures, embeddings, and vector search.
- Solid fundamentals in API design, relational databases and SQL, version control with Git, and CI/CD.
- Cloud experience in Azure, including managed services, identity, and secrets handling.
- Ability to reason about and debug non deterministic behavior, and the discipline to measure rather than assume that a system works.
- Clear written and verbal communication, including the ability to explain AI system limitations honestly to non technical stakeholders.
Preferred Qualifications
- Experience building agentic systems, multi step orchestration, or autonomous development tooling.
- Experience implementing or consuming Model Context Protocol servers.
- Experience with evaluation frameworks and LLM observability tooling.
- Prior work in healthcare, pharmacy, specialty pharmacy, hub services, or pharmaceutical manufacturer environments.
- Familiarity with HIPAA, HITRUST, SOC 2, or comparable compliance frameworks.
- Experience with Atlassian tooling (Jira, Confluence) and modern AI assisted development environments.
- Background in data engineering, platform engineering, or DevOps.
Level Expectations
- The AI Systems Developer owns assigned AI components end to end: design, implementation, evaluation, and operational readiness, with architectural direction from the Principal Architect and technical review from peers.
- Success is demonstrated by systems that behave predictably in production, evaluation evidence that supports the claims made about them, documentation another engineer can pick up, and sound judgment about where autonomy is appropriate and where it is not.
- Progression toward a senior or lead level is shown through ownership of platform level architecture, influence on engineering standards, and mentorship of other developers adopting AI tooling.
