Quality Assurance Engineer

Company Description

At Everite, we drive innovation through a blend of cutting-edge IT products, comprehensive services, expert staffing solutions, and transformative digital strategies. We deliver custom software, provide IT consulting, source top-tier talent, and guide businesses through digital transformation. Our mission is to help organizations achieve operational excellence and sustainable growth with tailored, technology-driven solutions.

About the Role:

We're looking for a meticulous QA Engineer to lead testing efforts for our multi-agent AI system. This is a cross-functional QA role where you’ll validate everything from AI-generated decisions to backend APIs, UI components, and third-party integrations (voice cloning, Twilio).

You’ll create and run test plans for 8+ autonomous agents, including LLM-backed agents, and help ensure every part of the system behaves reliably and ethically — before, during, and after deployment.

Key Responsibilities:

Manual & Exploratory Testing

• Understand agent workflows.
• Perform manual testing of workflows across:
• Web UI interfaces
• LLM responses and validation triggers
• Communications (email, SMS, voice)
• Backend API changes
• Identify issues in agent decision logic (false positives, incorrect matching, onboarding gaps, etc.).

Automated Testing

• Design and implement automated test scripts for:
• API endpoints
• Event/job creation flows
• Onboarding and contract handling
• SMS/Email/Voice triggers and responses
• Use tools like Postman, Selenium, Playwright, or PyTest for scripting.

Test Planning & Documentation

• Write and manage:
• Test cases and scenarios for each AI agent.
• Test Data and Synthetic data preparation (with support from ML team)
• Regression, functional, and smoke test plans.
• Edge-case tests for fallbacks, failed calls, no-shows, etc.
• Document known issues, test coverage, and agent-specific behavior.

QA Reporting

• Track bugs and raise detailed Jira tickets for:
• UI/UX inconsistencies
• API response mismatches
• Communication errors
• Model prediction issues (with support from ML team)
• Report test progress and bug severity regularly to the engineering team.

Collaboration & Feedback Loops

• Collaborate closely with:
• AI Engineers to validate LLM and ML behavior.
• Backend Devs for API consistency and error handling.
• Voice & Communication Engineers to validate cloned call quality and integrations.
• Help define agent success metrics from a QA lens (e.g., no incorrect crew matches, no duplicate invites, etc.)

Required Skills & Qualifications:

• 3–5+ years in QA Engineering or SDET roles
• Solid understanding of API and integration testing
• Experience with test automation tools:
• Postman / Swagger
• Selenium
• Python-based testing frameworks (PyTest preferred)
• Knowledge of version control (Git/Bitbucket) and issue tracking (Jira Issues)

Preferred (Bonus) Experience:

• Testing systems with AI/ML components or LLM-based applications
• Familiarity with voice API testing (e.g., Twilio Voice, ElevenLabs output validation)
• Worked on event-based or on-demand staffing platforms
• Understanding of CI/CD and QA automation pipelines

Tools You May Work With:

• Automation & API Tools: Postman, PyTest, Playwright, Selenium
• Bug Tracking: GitHub Projects, Jira
• Monitoring Feedback: Azure Monitor / Log Analytics
• Voice & Message Validation: Call transcripts, email/SMS message logs
• Other: Google Sheets (Test tracking), Teams (Internal team comms)

Qualifications

• Experience in Test Execution, Manual Testing, and Software Testing
• Proficiency in developing and executing Test Cases
• Strong Quality Assurance skills and ability to document defects effectively
• Excellent communication and collaboration skills
• Detail-oriented with strong analytical skills
• Knowledge of automation tools and scripting is a plus
• Bachelor's degree in Computer Science, Information Technology, or related field