A retrieval-augmented application can produce a fluent answer that is relevant to the question yet unsupported by the retrieved context. Groundedness evaluation focuses on that evidence relationship: does the response stay within what the supplied context can justify? Microsoft Foundry includes groundedness evaluators for RAG workflows, making this a direct concern for Microsoft AI-103<\/a> work on evaluation, monitoring, and fabrication detection.<\/p>\n The distinction matters inside Microsoft AI Agents<\/a>: relevance asks whether an answer addresses the user\u2019s need, while groundedness asks whether its claims are supported. An answer can be relevant and wrong, or grounded in context that is itself irrelevant to the question. Mature evaluation measures both.<\/p>\n Groundedness should be defined at claim level for serious systems. A response may contain five supported statements and one fabricated number; an overall fluent paragraph can hide that single high-impact failure. During dataset design, identify claims that require evidence and distinguish them from harmless connective language or general reasoning. This makes evaluator output easier to interpret and gives human reviewers a precise place to investigate when a score falls below threshold.<\/p>\n For a stored RAG interaction, evaluation should use the response together with the context passages that were supplied to generation; a query can also be included for richer scoring. If evaluation uses a different retrieval result from the one the model saw, it is no longer measuring the original answer path. Preserve retrieval artifacts in traces or evaluation datasets. knowledge grounding<\/a> depends on preserving source identity and context selection as part of the agent\u2019s observable behavior, not as disposable implementation detail.<\/p>\n Capture retrieval results with stable document identifiers, chunk text or hashes, ranking positions, and retrieval configuration. If the index is rebuilt before an evaluation rerun, the same query may return different evidence, making a generation regression look like a model change. Reproducible RAG evaluation therefore requires versioning the retrieval layer as carefully as the prompt and model. Store enough provenance to recreate what the generator actually received.<\/p>\n An answer can be perfectly supported by an outdated or incorrect document. Groundedness therefore depends on source governance. Retrieval should prefer current, authoritative material, enforce access controls, and preserve timestamps or version identifiers needed to distinguish current policy from superseded content.<\/p>\n Evaluation should include cases where two sources conflict or where a stale source contains a plausible answer. The RAG system must decide which evidence is authoritative before the model is asked to stay grounded in it.<\/p>\n A model can faithfully repeat an obsolete policy, a corrupted record, or a misleading document and still be perfectly grounded. Pair groundedness with source quality controls such as freshness, authority, ownership, and document-level trust. For high-impact workflows, retrieval should prefer canonical sources and expose provenance to the user. Evaluation can then separate two questions: did the model follow the evidence, and was the evidence appropriate for the decision?<\/p>\n If the correct evidence never entered context, a low groundedness or quality result may be blamed on the model even though retrieval caused the failure. Measure chunk retrieval, rank quality, and context coverage alongside answer groundedness. Existing guidance on RAG chunking<\/a> helps explain why an answer cannot cite a qualifier that was split away from the retrieved passage.<\/p>\n Create diagnostics that identify whether the reference source was indexed, retrieved, included, and then used correctly. This shortens the path from a bad score to the component that needs repair.<\/p>\n Use retrieval metrics to diagnose missing or badly ranked evidence before blaming the generator. Recall-oriented tests can ask whether the authoritative chunk appears in the candidate set; ranking tests can inspect whether it is buried below distracting passages. Generation tests should then hold evidence constant when comparing prompts or models. This decomposition prevents teams from endlessly tuning prompts when the real failure is that the required fact never reaches the model context.<\/p>\n This separation also changes the remediation path. Poor retrieval can require chunking, indexing, query-rewrite, metadata, or ranking changes, while unsupported generation may require prompt constraints, stronger refusal behavior, or a different model. Combining both into one score hides which subsystem owns the failure and makes regression work slower.<\/p>\n A response may contain five supported claims and one unsupported sentence. Aggregate scoring can hide that local failure, especially in long answers. For high-value workflows, evaluate at a claim or sentence level, or preserve human review tools that highlight which statements lack evidence.<\/p>\n This is particularly important when the unsupported sentence is a recommendation, exception, or numeric value. The most consequential claim may represent only a small fraction of the text.<\/p>\n Long answers deserve decomposition because one unsupported clause can be lost inside a strong aggregate score. Segment responses into claims or sentences, evaluate each against the supplied context, and preserve the lowest-scoring or highest-risk claims for manual review. The goal is not to maximize metric granularity for its own sake; it is to reveal where unsupported assertions concentrate so the product team can fix the relevant prompt, source, or workflow boundary.<\/p>\n For long answers, it is useful to preserve evidence spans or citations alongside claims so reviewers can inspect the exact support relationship rather than judging the answer as a single block. A response can be mostly correct and still contain one invented limit, date, or identifier that is operationally dangerous. That is why claim-level sampling remains valuable even when aggregate groundedness scores look healthy.<\/p>\n A RAG system should sometimes refuse to answer from insufficient context. Include evaluation cases where the index genuinely lacks the required evidence. The desired behavior may be to state that the sources do not support an answer, request clarification, or retrieve again\u2014not to produce a generally knowledgeable response from model memory.<\/p>\nThe evaluator needs the evidence that was actually available<\/h3>\n
A groundedness score does not prove the source is correct<\/h3>\n
Evaluate retrieval and generation separately<\/h3>\n
Sentence-level review can reveal partial fabrication<\/h3>\n
Groundedness needs unanswerable test cases<\/h3>\n