Job
Description
Job Area: Engineering Group, Engineering Group > Software Test Engineering General Summary: We are seeking a Engineer AI System-Level Test Engineer to lead end-to-end testing of Retrieval-Augmented Generation (RAG) AI systems for Hybrid, Edge-AI Inference solutions. This role will focus on designing, developing, and executing comprehensive test strategies for evaluating the reliability, accuracy, usability and scalability of large-scale AI models integrated with external knowledge retrieval systems. The ideal candidate needs to have deep expertise in AI testing methodologies, experience with large language models (LLMs), expertise in building test solutions for AI Inference stacks, RAG, search/retrieval architecture, and a strong background in automation frameworks, performance validation, and building E2E automation architecture. Experience testing large-scale generative AI applications, familiarity with LangChain, LlamaIndex, or other RAG-specific frameworks, and knowledge of adversarial testing techniques for AI robustness are preferred qualifications Key Responsibilities: Test Strategy & Planning Define end-to-end test strategies for RAG, retrieval, generation, response coherence, and knowledge correctness Develop test plans & automation frameworks to validate system performance across real-world scenarios. Hands-on experience in benchmarking and optimizing Deep Learning Models on AI Accelerators/GPUs Implement E2E solutions to integrate Inference systems with customer software workflows Identify and implement metrics to measure retrieval accuracy, LLM response quality Test Automation Build automated pipelines for regression, integration, and adversarial testing of RAG workflows. Validate search relevance, document ranking, and context injection into LLMs using rigorous test cases. Collaborate with ML engineers and data scientists to debug model failures and identify areas for improvement. Conduct scalability and latency tests for retrieval-heavy applications. Analyze failure patterns, drift detection, and robustness against hallucinations and misinformation. Collaboration Work closely with AI research, engineering teams & customer teams to align testing with business requirements. Generate test reports, dashboards, and insights to drive model improvements. Stay up to date with the latest AI testing frameworks, LLM evaluation benchmarks, and retrieval models. Required Qualifications: 1+ years of experience in AI/ML system testing, software quality engineering, or related fields. Bachelors or masters degree in computer science engineering/ data science / AI/ML Hands-on experience with test automation frameworks (e.g., PyTest, Robot Framework, JMeter). Proficiency in Python, SQL, API testing, vector databases (e.g., FAISS, Weaviate, Pinecone) and retrieval pipelines. Experience with ML model validation metrics (e.g., BLEU, ROUGE, MRR, NDCG). Expertise in CI/CD pipelines, cloud platforms (AWS/GCP/Azure), and containerization (Docker, Kubernetes). Minimum Qualifications: Bachelor's degree in Engineering, Information Systems, Computer Science, or related field. Applicants Qualcomm is an equal opportunity employer. If you are an individual with a disability and need an accommodation during the application/hiring process, rest assured that Qualcomm is committed to providing an accessible process. You may e-mail or call Qualcomm's toll-free number found . Qualcomm expects its employees to abide by all applicable policies and procedures, including but not limited to security and other requirements regarding protection of Company confidential information and other confidential and/or proprietary information, to the extent those requirements are permissible under applicable law. To all Staffing and Recruiting Agencies Please do not forward resumes to our jobs alias, Qualcomm employees or any other company location. Qualcomm is not responsible for any fees related to unsolicited resumes/applications. If you would like more information about this role, please contact .
