Why this matters
An AI feature can produce several acceptable answers to the same input. A strict text comparison therefore creates false failures, while an overly tolerant test may accept unsafe or irrelevant output. The test target must shift from exact wording to measurable qualities and boundaries.
The team should agree what “good enough” means for the user: relevance, factual grounding, safety, format, latency and consistency may all matter, but not always with the same weight.
How to do it well
Define the user outcome
Describe what the answer must help the user achieve, not just what text the model should produce.
Create an evaluation rubric
Translate quality into observable criteria such as relevance, completeness, factual support, tone, format and safety.
Build a representative test set
Include common cases, edge cases, ambiguous prompts, adversarial inputs and examples from real usage.
Separate hard rules from scored qualities
Validate required JSON, prohibited content and mandatory fields deterministically; score open-ended qualities with thresholds.
Repeat and compare
Run important prompts several times and across model or prompt versions to measure variability and regression.
Review samples with people
Use calibrated reviewers for borderline cases and regularly compare human judgement with automated evaluation.
What to avoid
- Expecting one golden sentence as the only correct answer.
- Testing only friendly and well-formed prompts.
- Changing model, prompt and retrieval data simultaneously without isolating the cause.
- Using an AI judge without checking its bias and agreement with human reviewers.
- Ignoring latency, cost and failure behaviour when upstream services are unavailable.
Practical example
For an AI-generated campaign summary, hard checks can require the correct campaign identifiers and forbid unsupported numeric claims. A rubric can then score clarity, relevance and actionable recommendations.
A release comparison should run the same dataset against both versions, inspect score distributions and manually review examples with the largest change rather than relying on one average.
Lesson for practice
Testing non-deterministic systems is disciplined evaluation, not the abandonment of expectations. Clear criteria, diverse data, repeated runs and human calibration make variability manageable.
A good practice does not have to be complicated. It should be intentional, repeatable and explainable: the team should understand why the control exists, what evidence it provides and how feedback will improve the next iteration.