AI Automation Test Engineer

Position Overview

We are seeking a passionate AI Automation Test Engineer to join our innovative UK based stealth startup, in Bengaluru. This role offers an exciting opportunity to shape the testing strategy for cutting-edge AI products while working in a fast-paced, dynamic environment. You'll be instrumental in ensuring the quality and reliability of our AI-driven solutions through comprehensive automated testing frameworks.

Key Responsibilities

AI/ML Testing

• Design and implement automated testing frameworks specifically for AI/ML models and systems
  
• Develop test strategies for model performance, accuracy, bias detection, and robustness
  
• Create automated pipelines for data validation, model training verification, and inference testing
  
• Implement A/B testing frameworks for AI model comparison and evaluation
  

Test Automation Development

• Build and maintain end-to-end test automation suites using modern testing frameworks
  
• Develop API testing automation for microservices and ML model endpoints
  
• Create performance and load testing automation for AI inference systems
  
• Design automated regression testing for continuous integration/deployment pipelines
  

Quality Assurance

• Collaborate with AI/ML engineers to define testing requirements and acceptance criteria
  
• Establish quality gates and metrics for AI model deployment
  
• Conduct exploratory testing of AI features and user experiences
  
• Monitor and analyse test results, providing actionable insights to development teams
  

Infrastructure & DevOps

• Set up and maintain testing environments and test data management systems
  
• Integrate automated tests with CI/CD pipelines using tools like Jenkins, ArgoCD, or GitHub Actions
  
• Implement monitoring and alerting for test execution and system health
  
• Manage test data pipelines and synthetic data generation for AI model testing
  

Required Qualifications

Technical Skills

• Programming: Proficiency in Python, with experience in testing frameworks (pytest, unittest, Robot Framework)
  
• AI/ML Testing: Understanding of machine learning concepts, model evaluation metrics, and testing methodologies
  
• Automation Tools: Hands-on experience with Selenium, Appium, or similar web/mobile automation tools
  
• API Testing: Experience with REST/GraphQL API testing using tools like Postman, Newman, or requests library
  
• Version Control: Proficient with Git and collaborative development workflows
  

Experience Requirements

• 3-6 years of experience in test automation and quality assurance
  
• 1-2 years of experience testing AI/ML systems or data-driven applications
  
• Experience with cloud platforms (AWS, Azure, or GCP) and containerization (Docker, Kubernetes)
  
• Familiarity with databases (SQL and NoSQL) and data validation techniques
  

Preferred Qualifications

• Experience with MLOps tools and practices (MLflow, Kubeflow, or similar)
  
• Knowledge of performance testing tools (JMeter, Locust, or K6)
  
• Understanding of data science workflows and model lifecycle management
  
• Experience with monitoring tools (Prometheus, Grafana, ELK stack)
  
• Background in statistics or data analysis
  

What We Offer

Compensation & Benefits

• Competitive salary up to ₹15,00,000 per annum
  
• Performance-based bonuses and annual increments
  
• Comprehensive health insurance for you and your family
  

Professional Growth

• Opportunity to work with cutting-edge AI technology in stealth mode
  
• Direct impact on product development and company direction
  
• Mentorship from experienced AI and engineering leaders
  
• Learning budget for courses, conferences, and certifications
  

Work Environment

• Modern office space in Bengaluru with all necessary amenities
  
• Collaborative, innovation-driven culture
  
• Regular team events and knowledge-sharing sessions
  

About the Role

As an early-stage team member, you'll have the unique opportunity to build testing practices from the ground up, directly influence product quality, and grow with the company. This position is ideal for someone who thrives in ambiguous environments, enjoys solving complex technical challenges, and wants to be part of building something revolutionary in the AI space.

Note: Due to our stealth mode status, specific product details will be shared during the interview process with qualified candidates who sign appropriate confidentiality agreements.

Application Process

Interested candidates should submit their resume along with:

• Brief cover letter explaining interest in AI testing and startup environments
  
• Any relevant certifications or achievements in testing or AI/ML domains

We are an equal opportunity employer committed to diversity and inclusion. All qualified applicants will receive consideration regardless of race, gender, age, religion, sexual orientation, or disability status.