AI/ ML Test Engineer

Job Description

Technical Skills & Experience: Programming Skills: Proficiency in Python, capable of developing AI-driven test scripts. Test Automation: Solid experience in test automation practices. Model Evaluation: Hands-on experience evaluating and testing Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) systems, and computer vision models. Evaluation Frameworks: Skilled in designing evaluation frameworks to assess AI model performance, accuracy, bias, and safety. Familiarity with open-source evaluation tools such as DeepEval and similar frameworks. Prompt Engineering & Metrics: Knowledge of prompt engineering and LLM evaluation metrics like perplexity, BLEU, ROUGE, and human evaluation techniques. RAG System Testing: Experience in testing RAG systems, including retrieval quality assessment and hallucination detection. Computer Vision Testing: Familiarity with vision model testing methodologies and image processing evaluation. CI/CD Integration: Working knowledge of Continuous Integration/Continuous Deployment (CI/CD) tools like GitLab, Jenkins, or Azure DevOps. Preferred Qualifications Evaluation Framework Proficiency: Hands-on experience with at least one open-source AI evaluation framework, such as DeepEval, is essential. Vector Databases & Embedding Evaluation: Familiarity with vector databases like FAISS or Milvus, and experience in evaluating embedding models for tasks such as similarity search and retrieval. Multimodal AI Systems: Understanding of AI systems that process multiple data modalities (e.g., text, images) and the unique challenges involved in their evaluation. Human-in-the-Loop Evaluation: Experience incorporating human feedback into AI evaluation processes to enhance model performance and reliability Perks and benefits Best of Technology Flexible Working Hours Rewards and Recognition Medical Insurance Round the Clock Snacks Paid Time-Off and Vacation