Data Annotation Engineer

Role Overview for Engineer REQs

An AI Testing Engineer for Windows is responsible for validating AI-driven software components, frameworks, and models integrated into Windows-based systems. This role ensures quality, reliability, and compliance of AI features through automated and manual testing, leveraging advanced AI-powered testing methodologies.

Key Responsibilities
• Test Case Design & Automation Develop and maintain automated test frameworks using AI-assisted tools. Implement context-based AI test case generation leveraging LLMs and Retrieval-Augmented Generation (RAG) for Windows-specific scenarios. • AI Model & Feature Validation Validate machine learning models, NLP algorithms, and generative AI features integrated into Windows applications. Perform adversarial robustness testing and trustworthiness checks as per OWASP AI Testing Guide standards. • Regression & Gap Analysis Conduct AI-driven test escape and gap analysis to identify coverage gaps and recommend enhancements for Windows test suites. • Performance & Stress Testing Execute performance and stress tests for AI Apps, ensuring scalability and reliability. • Debugging & RCA Analyze logs, and failure signatures to triage issues and optimize test workflows.

Required Skills
• Strong programming skills in Python, C++, and scripting languages. • Experience with Windows platform testing, including debugging tools (WinDBG, JTAG) • Familiarity with AI frameworks (TensorFlow, PyTorch) and Windows ML. • Knowledge of test automation frameworks and AI-powered testing tools. • Understanding of MLOps, semantic search, and vector databases for test data management.

Preferred Qualifications
• Exposure to Generative AI (GenAI), LangChain, and custom LLM development for test case generation.• Certifications in ISTQB, AI Testing, or related QA domains