A hallucination test is only useful when the team can define what the model was expected to know and which evidence it was allowed to use. In retrieval-augmented generation, a response can be wrong because the model ignored good evidence, because retrieval supplied weak evidence, because the source itself was outdated, or because the question had no supported answer. Treating all four cases as “the model hallucinated” hides the engineering layer that actually needs to change.
Hallucination evaluation is central to Amazon AWS AIP-C01, which includes model evaluation and validation, RAG, responsible AI, content safety, monitoring, and troubleshooting. In an AWS generative AI system, Amazon Bedrock evaluations and Guardrails can contribute useful signals, but the application still needs a domain-specific test set, evidence lineage, release thresholds, and incident feedback loops.
The strongest evaluation program therefore measures groundedness at several levels. It asks whether retrieval found the right source, whether the answer remained faithful to that source, whether the answer actually addressed the user’s question, and whether the system abstained when the evidence was insufficient. Those dimensions lead to better fixes than a single generic “quality” score.
Define hallucination against an evidence contract
For a closed-book support assistant, the evidence contract may say that every factual claim about a product must be supported by approved documentation retrieved during the request. For a general assistant, the contract may allow model knowledge for low-risk background facts while requiring citations for financial, legal, or operational claims. The acceptable behavior depends on the product.
Once the contract is explicit, evaluators can label failure precisely. A statement may be unsupported by retrieved evidence, contradicted by the source, irrelevant to the question, outdated relative to the current document version, or overly certain when the source is ambiguous. These labels point toward retrieval, prompt, source-governance, or generation changes.
AI evaluation pipelines need multiple metrics because hallucination, retrieval failure, unsupported claims, safety failures, and task errors are different failure classes. Evaluation should preserve enough evidence to diagnose why a score changed instead of only reporting that a model version is “better.”
Test retrieval before blaming the generator
A RAG system cannot produce a grounded answer from evidence it never received. Build retrieval-only tests with questions mapped to expected source passages or documents. Measure whether relevant evidence appears in the candidate set, whether important evidence is missing, and how much irrelevant context is included. Retrieval quality can regress after chunking changes, embedding migrations, source updates, or metadata-filter modifications even when the generation prompt is unchanged.
For multi-hop questions, the expected evidence may contain more than one passage. A question about a permission change and its audit effect may require both an authorization document and a logging document. A retrieval metric that treats one relevant chunk as success can miss incomplete evidence. Test the coverage needed for the task.
Also test no-answer cases. The system should recognize when the approved corpus does not contain the requested fact. If every query is assumed to have an answer, the evaluation set rewards guessing rather than calibrated refusal.
Use faithfulness and relevance as separate signals
Amazon Bedrock RAG evaluation includes metrics that distinguish the quality of retrieved context from the quality of generated responses. Faithfulness measures whether a generated response stays aligned with retrieved text, while relevance asks whether the response addresses the user’s query. A response can be faithful but irrelevant, or relevant-sounding but unsupported.
That separation is operationally valuable. Low faithfulness with good retrieval points toward generation instructions, context construction, or model behavior. Low retrieval relevance with faithful generation points toward search, chunking, filters, or the corpus. Improving the wrong layer can increase cost without improving the user experience.
LLM judge models should treat evaluator output as structured evidence rather than unquestioned truth, because the judge can introduce its own variability and bias. Judge models have their own variability and biases, so high-impact decisions benefit from calibrated rubrics, examples, and periodic human review.
Contextual grounding checks are runtime controls, not a full benchmark
Amazon Bedrock Guardrails can perform contextual grounding checks when a grounding source, user query, and model response are available. The check scores grounding and relevance and can filter responses below configured thresholds. This can be valuable as a runtime safeguard for supported use cases, especially when the application wants a deterministic policy action around obviously ungrounded output.
A runtime threshold should be tuned against representative traffic. A threshold that is too low may allow unsupported content; a threshold that is too high may block useful paraphrases or edge cases. Evaluate false positives and false negatives separately, and remember that one global threshold may not fit every task. A compliance summary and a creative rewriting feature may need different policies.
Grounding checks also have scope boundaries. Teams should not assume that one guardrail feature evaluates every conversational pattern or every possible factual error. The evaluation program still needs offline datasets and domain-specific review.
Build test sets from real failure modes
Generic benchmark prompts are useful for model capability, but application hallucinations are often caused by the organization’s own data. Create test cases around duplicated policies, conflicting source versions, tables with important headers, abbreviations, access-controlled documents, ambiguous entity names, long procedures, sparse retrieval results, and questions whose correct answer is “not documented.”
Production incidents are especially valuable additions. When an unsupported answer escapes, convert the scenario into a regression case after removing or protecting sensitive details. Record the source state and expected behavior so the same defect cannot silently return after a model, prompt, or retrieval change.
A disciplined LLM regression-testing process treats these cases like software tests: they have stable inputs, expected properties, versioned datasets, release thresholds, and ownership when they fail. The expected output does not always need to be one exact sentence; it can specify required claims, prohibited claims, citation requirements, and acceptable refusal behavior.
Evaluate the answer at claim level when risk is high
A whole-response score can hide one dangerous sentence inside an otherwise good answer. For high-risk domains, break responses into factual claims and assess whether each material claim is supported by the provided evidence. This is more expensive than assigning one overall grade, but it creates a clearer link between the source and the statement the user will act on.
Citation evaluation adds another layer. A response may cite a source that is generally related but does not support the adjacent claim. Measure citation precision and coverage when citations are a product feature. The user should not be given the impression of verification merely because a source link is present somewhere in the answer.
Claim-level review also exposes hedging problems. A source might say that a feature is available in selected Regions, while the answer says it is universally available. Most of the sentence is related to the evidence, but the quantifier changes the operational meaning. Rubrics should detect that kind of overstatement.
Use multiple evaluators for release decisions
Automatic metrics are scalable, judge models can handle nuanced rubrics, and humans can resolve ambiguous or high-impact cases. They should complement rather than replace one another. A fast automated suite can run on every change; a judge-based suite can score broader qualitative dimensions; sampled human review can calibrate whether those scores match expert expectations.
For model comparisons, keep the dataset and generation configuration stable enough that results are interpretable. If the prompt, retriever, model, temperature, and source corpus all change at once, a score movement cannot be attributed. Controlled experiments reduce the temptation to declare a model “better” because one demo improved.
Confidence intervals and segment results matter when the dataset is small. A two-point average improvement may come entirely from one easy category while safety-critical questions got worse. Report results by task type, risk class, language, source type, and other dimensions that matter to the product.
Release thresholds should reflect consequence, not a universal pass rate
Not every hallucination carries the same cost. An unsupported recommendation in a brainstorming feature is different from an invented configuration value that an operator may deploy. Define higher thresholds and stronger review for categories where an incorrect claim can change money, permissions, compliance, or production systems. Lower-risk creative tasks can tolerate more variation as long as the product does not misrepresent the output as verified fact.
Release policy should also distinguish regressions from known limitations. A model change that improves average quality but worsens a critical slice should not pass simply because the overall score increased. Keep explicit blocking cases for the behaviors the product cannot accept, and require investigation when those cases fail.
Production monitoring closes the evaluation loop
Offline testing cannot predict every user query. Runtime telemetry should identify low grounding scores, refusals, retrieval misses, user corrections, repeated rephrasing, escalations, and other signals that an answer may have failed. Sampling those cases for reviewed evaluation expands the test set toward the distribution the product actually sees.
Within the Amazon AWS platform, evaluation outputs, model invocation logs, CloudWatch metrics, traces, and security logs can be correlated around a stable request identifier. Keep the raw content protected, but preserve enough metadata to connect a bad answer with the model version, prompt version, retrieval configuration, source set, guardrail version, and evaluator results.
A hallucination program is mature when it can answer more than “How often is the model wrong?” It should show which layer failed, which query classes are deteriorating, which release introduced the change, and whether runtime safeguards are catching unsupported output. That turns hallucination from an abstract model concern into a measurable reliability property of the complete AI system.
