2mo ago

AI Quality Engineer (Contract)

India·HyderabadContractmid

EngineeringQuality Engineer

4 views0 saves0 applied

Technical Tools

EngineeringQuality Engineer

~1 min read

→Test and validate AI-generated insights, recommendations, and decision-making workflows.
→Evaluate LLM and RAG systems for accuracy, relevance, consistency, factuality, and hallucinations.
→Validate retrieval quality, context relevance, grounding, and response quality in RAG systems.
→Test AI agents and autonomous workflows across functional, negative, and edge-case scenarios.
→Define AI evaluation criteria, test datasets, quality metrics, and validation processes.
→Perform regression testing for models, prompts, RAG configurations, and AI workflows.
→Collaborate with AI/ML engineers to identify issues and improve AI system quality.

~1 min read

Strong understanding of AI/ML and Generative AI testing.
Hands-on experience testing LLM and RAG-based applications.
Knowledge of LLM evaluation, hallucination detection, relevance, and response quality.
Understanding of AI agents and recommendation systems.
Strong analytical and problem-solving skills.

Experience with PyTest and automated testing frameworks.
Experience building automated AI evaluation and regression frameworks.
Familiarity with tools such as RAGAS, DeepEval, LangSmith, or equivalent.
Experience with CI/CD-based test automation, performance testing, or AI guardrails.
Familiarity with cloud platforms (AWS/Azure/GCP) and observability tools.