Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in hyderabad
>
Opentext
>
Sr. Quality Assurance Engineer- AI

Sr. Quality Assurance Engineer- AI

Opentext

7 - 12 years

13 - 18 Lacs

hyderabad

Posted:4 days ago| Platform:

Apply

Skills Required

quality assurance kubernetes rest python regression testing ai aws sagemaker hr llm microservices docker qa automation gcp grafana regression jenkins gitlab data privacy prometheus flask aws architecture azure

Work Mode

Work from Office

Job Type

Full Time

Job Description

YOUR IMPACT

Quality Assurance (QA) Engineer

validating Generative AI systems

AI engineers, data scientists, and product owners

What The Role Offers

Be part of a
next-generation AI engineering team
delivering enterprise-grade GenAI solutions.
Gain hands-on experience testing
LangGraph-based agentic workflows
and
RAG pipelines
.
Learn from
senior AI engineers
working on production-grade LLM systems.
Opportunity to grow into
AI Quality Specialist
or
AI Evaluation Engineer
roles as the team expands.

Develop and execute
test cases
for validating RAG pipelines, LLM integrations, and agentic workflows.
Validate
context retrieval accuracy
,
prompt behaviour
, and
response relevance
across different LLM configurations.
Conduct
functional, integration, and regression testing
for GenAI applications exposed via APIs and microservices.
Test
Agent-to-Agent (A2A)
&
Model Context Protocol (MCP)
communication flows for correctness, consistency, and task coordination.
Verify
data flow and embedding accuracy
between vector databases (Milvus, Weaviate, pgvector, Pinecone).

Build and maintain
automated test scripts
for evaluating AI pipelines using
Python and PyTest
.
Leverage
LangSmith
,
Ragas
, or
TruLens
for automated evaluation of LLM responses (factuality, coherence, grounding).
Integrate AI evaluation tests into
CI/CD pipelines
(GitLab/Jenkins) to ensure continuous validation of models and workflows.
Support
performance testing
of AI APIs and RAG retrieval endpoints for latency, accuracy, and throughput.
Assist in creating
automated reports
summarizing evaluation metrics such as Precision@K, Recall@K, grounding scores, and hallucination rates.

Validate
guardrail mechanisms
,
response filters
, and
safety constraints
to ensure secure and ethical model output.
Use
OpenTelemetry (OTEL)
and
Grafana dashboards
to monitor workflow health and identify anomalies.
Participate in
bias detection and red teaming
exercises to test AI behavior under adversarial conditions.

Work closely with
AI engineers
to understand system logic, prompts, and workflow configurations.
Document test plans, results, and evaluation methodologies for repeatability and governance audits.
Collaborate with
Product and MLOps teams
to streamline release readiness and model validation processes.

What You Need To Succeed

Education:
Bachelor??s degree in Computer Science, AI/ML, Software Engineering, or related field.
Experience:
4??7 years in Software QA or Test Automation, with at least
2 years exposure to AI/ML or GenAI systems
.
Solid hands-on experience with
Python
and
PyTest
for automated testing.
Basic understanding of
LLMs
,
RAG architecture
, and
vector database operations
.
Exposure to
LangChain
,
LangGraph
, or other
agentic AI frameworks
.
Familiarity with
FastAPI
,
Flask
, or REST API testing tools (Postman, PyTest APIs).
Experience with
CI/CD pipelines
(GitLab, Jenkins) for test automation.
Working knowledge of
containerized environments
(Docker, Kubernetes).
Understanding of
AI evaluation metrics
(Precision@K, Recall@K, grounding, factual accuracy).

Exposure to
AI evaluation frameworks
like
Ragas
,
TruLens
, or
OpenAI Evals
.
Familiarity with
AI observability
and telemetry tools (OpenTelemetry, Grafana, Prometheus).
Experience testing
LLM-powered chatbots
,
retrieval systems
, or
multi-agent applications
.
Knowledge of
guardrail frameworks
(Guardrails.ai, NeMo Guardrails).
Awareness of
AI governance principles
,
data privacy
, and
ethical AI testing
.
Experience with
cloud-based AI services
(AWS Sagemaker, Azure OpenAI, GCP Vertex AI).
Curious and eager to learn emerging AI technologies.
Detail-oriented with strong problem-solving and analytical skills.
Excellent communicator who can work closely with engineers and product managers.
Passion for quality, reliability, and measurable AI performance.
Proactive mindset with ownership of test planning and execution.#LI-MD1

More Jobs at Opentext

Sr. Software Engineer

Hyderabad

7 - 13 yrs

INR 9 - 15 Lacs

Technical Marketing Manager

Bengaluru

12.0 - 17.0 yrs

INR 8 - 12 Lacs

Lead Software Engineer

Bengaluru

5.0 - 10.0 yrs

INR 25 - 30 Lacs

Sr. Software Engineer

Bengaluru

5.0 - 10.0 yrs

INR 6 - 10 Lacs

Quality Assurance and Automation Engineer

Bengaluru

3.0 - 8.0 yrs

INR 5 - 15 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.