Artificial Intelligence Engineer

About Certa

Certa (getcerta.com) is a Silicon Valley-based startup automating vendor, supplier, and stakeholder onboarding processes for businesses globally. Serving Fortune 500 and Fortune 1000 clients, Certa's engineering team tackles expansive and deeply technical challenges, driving innovation in business processes across industries.

Role Overview

AI Engineer

Key Responsibilities

• Design and Develop AI Features:

  Lead the design, development, and deployment of generative AI capabilities and LLM-powered services that deliver engaging, human-centric user experiences. This includes building features like intelligent chatbots, AI-driven recommendations, and workflow automation.

• RAG Pipeline Implementation:

  Design, implement, and continuously optimize end-to-end RAG (Retrieval-Augmented Generation) pipelines, including data ingestion, document chunking, vector indexing, and prompt engineering strategies. Ensure our AI systems can efficiently retrieve and use information from knowledge bases to enhance answer accuracy.

• Build LLM-Based Agents:

  Develop and refine LLM-based agentic systems that can autonomously perform complex tasks or assist users in multi-step workflows. Incorporate tools for planning, memory, and context management (e.g., long-term memory stores, tool use via APIs) to extend our AI agents' capabilities.

• Integrate with Product Teams:

  Work closely with product managers, designers, and other engineers to integrate AI capabilities seamlessly, ensuring features align with user needs and business goals. You'll collaborate cross-functionally to translate requirements into AI solutions and iterate based on feedback.

• System Evaluation & Iteration:

  Rigorously evaluate the performance of AI models and pipelines using appropriate metrics—including accuracy, response latency, and avoidance of errors like hallucinations. Conduct thorough testing and use user feedback to drive continuous improvements.

• Code Quality & Best Practices:

  Write clean, maintainable, and testable code while following software engineering best practices. Ensure AI components are well-structured, scalable, and fit our overall system architecture. Implement monitoring and logging for AI services to track performance in production.

• Mentorship and Knowledge Sharing:

  Provide technical guidance and mentorship to team members on best practices in generative AI development. Help educate colleagues through code reviews and tech talks in areas like prompt engineering and evaluating model outputs. Foster a culture of continuous learning and experimentation.

• Research & Innovation:

  Continuously explore the latest advancements in AI/ML (new models, libraries, techniques) and assess their potential value. You'll have the freedom to prototype innovative solutions and bring them into our platform if they prove beneficial. Staying current with emerging research and industry trends is a key part of this role.

Required Skills and Qualifications

• Software Engineering Experience:

  3+ years (Mid-level) / 5+ years (Senior) of professional software engineering experience. You'll need rock-solid backend development skills with expertise in

  Python

  and designing scalable APIs/services. Experience building and deploying systems on

  AWS

  or similar cloud platforms is a must, including strong system design abilities with a track record of building robust, maintainable architectures.

• LLM/AI Application Experience:

  Proven experience building applications that leverage large language models or generative AI. You've spent time prompting and integrating language models into real products (e.g., chatbots, semantic search, AI assistants) and understand their behavior and failure modes. Demonstrable projects or work in LLM-powered application development—especially using techniques like RAG or building LLM-driven agents—will make you stand out.

• AI/ML Knowledge:

  Prioritize applied LLM product engineering over traditional ML pipelines. You should have a strong understanding of prompt design, function calling/structured outputs, tool use, and the RAG components that matter (parsing, chunking, metadata, re-ranking, embedding, model selection). Make pragmatic model choices based on latency, cost, and safety trade-offs. Design robust evaluation systems (e.g., golden sets, A/B tests) and track production proxies like retrieval recall and hallucination rate. Solid grasp of embeddings, tokenization, vector search fundamentals, and familiarity with agent patterns (planning, tool orchestration, memory) and guardrail techniques.

• Tooling & Frameworks:

  Hands-on experience with the AI/LLM tech stack and libraries. This includes proficiency with LLM orchestration libraries like

  LangChain, LlamaIndex,

  etc. Experience working with vector databases or semantic search (e.g., Pinecone, Chroma, Milvus) to enable retrieval-augmented generation is highly desired.

• Cloud & DevOps:

  Own the productionization of LLM/RAG-backed services on AWS (e.g., ECS/EKS/Lambda, API Gateway, S3, DynamoDB, SQS/SNS, VPC) and with infrastructure-as-code (Terraform/CDK). You're comfortable shipping stateless APIs and event-driven pipelines with strong observability, security, and progressive delivery. You manage token/cost budgets, apply caching/batching/streaming, and implement graceful fallbacks.

• Product and Domain Experience:

  Experience building enterprise (B2B SaaS) products is a strong plus. This means you understand considerations like user experience, scalability, security, and compliance.

• Strong Communication & Collaboration:

  Excellent interpersonal and communication skills, with an ability to explain complex AI concepts to non-technical stakeholders. You work effectively in cross-functional teams to drive projects forward.

• Problem-Solving & Autonomy:

  Self-motivated and able to manage multiple priorities in a fast-paced environment. You have a demonstrated ability to troubleshoot complex systems and quickly prototype solutions. A “figure it out” attitude is key.

Preferred (Bonus) Qualifications

• Full-Stack & Frontend Skills:

  While this is a backend/AI role, having a deep full-stack background (especially modern frontend frameworks or Node.js) is beneficial.

• Advanced AI Techniques:

  Familiarity with advanced techniques like fine-tuning LLMs (e.g., using LoRA adapters), reinforcement learning from human feedback (RLHF), or other emerging ML methodologies. Experience working with open-source LLMs (LLaMA, Mistral, etc.) is a plus.

• Multi-Modal and Agents:

  Experience developing complex agentic systems (e.g., multi-agent systems, integrating LLMs with tool networks) is a bonus. Knowledge of multi-modal AI could also be useful.

• Startup/Agile Environment:

  Prior experience in an early-stage startup or similarly fast-paced environment where you’ve worn multiple hats and adapted to rapid changes.

• Community/Research Involvement:

  Active participation in the AI community (open-source contributions, research publications, or blogging) is appreciated.

Perks:

• Best-in-class compensation
• Fully-remote work with flexible schedules
• Continuous learning
• Massive opportunities for growth
• Yearly offsite
• Quarterly hacker house
• Comprehensive health coverage
• Parental Leave
• Latest Tech Workstation
• Rockstar team to work with (we mean it!)