Gen AI Developer

Requirements

GenAI (Core)

• Python-first implementation: Build LLM/RAG services in Python (FastAPI/asyncio, Pydantic), with clean APIs and tests.
• Agentic AI workflows: Tool-using agents (planning, memory, multi-step execution), function/tool calling, and recovery/fallback paths.
• Streaming responses: Server-streamed tokens from the model to the UI.
• RAG pipelines: Ingestion, chunking, embeddings, indexing, reranking; grounded answers with citations (pgvector/FAISS/Pinecone).
• Prompt management: Versioning, templates/params, safe fallbacks and rollback.
• Evaluations: Hallucination/groundedness checks, small regression suite for prompts/retrievers.
• Guardrails: PII detection/redaction, content safety, domain constraints.
• Perf & cost: Context reduction, caching/batching, request pacing/rate-limit handling.
• Multimodal (basic): Text + files; ASR/TTS as a plus.
• Telemetry: Token usage, grounding rate, answer length, latency, errors/timeouts.

Integration (Core)

• API layer: Design/ship REST (or gRPC) endpoints that orchestrate tools, retrieval, and post-processing (citations/formatting).
• Streaming UX: SSE/WebSockets with retries, backoff, partial responses, and user cancellation.
• MCP (Model Context Protocol): Implement/consume MCP tool adapters (files, web, DB/connectors), capability negotiation, auth/permissions, and resource limits for safe tool access.
• Vector stores & data sources: Integrate pgvector/FAISS/Pinecone plus DB/file stores and enterprise connectors (e.g., SharePoint/Confluence/Slack/Jira).
• AuthN/Z & tenancy: JWT/OAuth, role/tenant isolation, secrets management, audit logging for AI actions.
• Observability: Logs, metrics, traces; dashboards/alerts for latency, error rate, token spend.
• Queues/workflows: Background jobs for ingestion/summarization with idempotency and retries.
• Data governance: Input/output validation, schema contracts, PII handling, retention policies.
• CI/CD: For prompts, retrieval configs, and API changes; blue/green or canary releases.