AI/ML

Job Overview

We are seeking a Senior AI Test Automation Engineer to lead the quality assurance efforts for AI/ML-powered applications and systems. This role will be critical in designing and implementing test strategies that ensure the accuracy, robustness, fairness, and reliability of AI models and ML pipelines. The ideal candidate will combine deep experience in test automation frameworks with a strong understanding of AI/ML workflows and data science practices.Key ResponsibilitiesDesign, develop, and maintain automated test frameworks and test suites for AI/ML systems, APIs, and data pipelines.Validate the integrity, performance, and accuracy of machine learning models in both development and production environments.Implement test coverage for edge cases, fairness, bias, explainability, and model drift in ML workflows.Collaborate with data scientists and ML engineers to review model assumptions, inputs, and outputs.Create synthetic and real-world test data to evaluate AI/ML model behavior under various conditions.Monitor CI/CD pipelines to ensure models and services are thoroughly tested before deployment.Apply data validation and statistical testing to verify training data quality and results consistency.Drive root cause analysis of model failures and identify opportunities to improve test effectiveness.Lead performance benchmarking and regression testing for ML services at scale.

Primary Skills (Must-Have)

Test Automation Expertise in tools like PyTest, Robot Framework, Selenium, Postman, or REST Assured.Python Programming Strong experience with Python, particularly in writing test scripts for ML/AI services.ML/AI Knowledge Understanding of machine learning lifecycle, supervised/unsupervised models, and common ML frameworks (e.g., scikit-learn, TensorFlow, PyTorch).Data Validation Experience with tools/libraries like Great Expectations, Pandera, or custom Python-based validation.CI/CD Integration Proficiency with Jenkins, GitLab CI, or similar tools for integrating automated tests in pipelines.API Testing Proven ability to test RESTful APIs and ML endpoints.Model Evaluation Metrics Familiarity with metrics like precision, recall, F1 score, confusion matrix, AUC-ROC.Secondary Skills (Nice-to-Have)Cloud Platforms Familiarity with AWS, Azure, or GCP for testing AI services deployed on cloud infrastructure.ML Monitoring Tools Exposure to tools like Evidently AI, MLflow, or Weights & Biases.Bias/Fairness Testing Knowledge of ethical AI practices and tools like Fairlearn or Aequitas.Containerization Experience with Docker/Kubernetes to manage test environments.Performance Testing Tools like JMeter, Locust, or k6.Version Control Git/GitHub for codebase management.SQL and Data Engineering Basics For testing data pipelines and verifying large datasets.

Qualifications

Bachelor or Master degree in Computer Science, Data Science, Engineering, or related field.5+ years of experience in test automation with at least 2+ years working in AI/ML environments.Demonstrated ability to work in cross-functional agile teams with ML engineers and data scientists.Strong analytical, problem-solving, and communication skills