Break each answer into claims and check each against the passage it cites. Does the source exist in the case, is it the current version, and does it say what the agent says it says? Include cases where the sources are silent or contradict each other, because the grounded answer there is to say so. Score uncited claims as failures.
What does grounded mean, and who owns the test?
An answer is grounded when every factual claim in it can be traced to a source the agent was given for that case: a policy document, a record in a system, a customer's letter. Grounding is what lets a reviewer check the agent's work and what lets the firm stand behind an answer. The grounded gate is one of the seven, and it must clear its floor on its own.
Grounded does not mean right. A claim faithfully drawn from an out-of-date policy is grounded and wrong. That distinction is the subject of grounded is not the same as correct, and it shapes how the test is built.
The capability owner owns the test. The owners of the source documents should help, because they know which passages are commonly misread.
What are the steps?
- Person, owner: defines the sources. Which documents and systems count as sources for this capability, and how the agent must cite them: document, version and passage.
- Person: builds ordinary cases. Filed cases where the answer is in the sources.
- Person: builds silent cases. Cases where the sources do not answer the question. The grounded answer is to say the sources do not cover it, and to escalate if the capability requires.
- Person: builds conflict cases. Two sources that disagree, such as a policy and a product sheet. The grounded answer names the conflict.
- Person: builds version cases. An old and a current version both present. The grounded answer uses the current one.
- Agent, replay only: answers every case.
- Platform: splits answers into claims. Each factual statement on its own.
- Platform and scorers: check each claim. The cited source exists in the case; it is the right version; the cited passage supports the claim, not just mentions the topic. Uncited factual claims fail.
- Scorers: check provenance. A claim resting on an external or declared source presented as if it were the firm's own record is a grounding failure.
- Person, owner: reviews failures and adds them to the set.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Define sources | Nothing | None | Choosing sources |
| Ordinary cases | Nothing | None | Selecting them |
| Silent cases | Nothing | None | Designing them |
| Conflict cases | Nothing | None | Designing them |
| Version cases | Nothing | None | Designing them |
| Replay | Answers with citations | Replay only | None |
| Split claims | Nothing | None | None |
| Check claims | Nothing | None | Judging support |
| Provenance | Nothing | None | Judging class use |
| Review | Nothing | None | Adding failures |
What controls make it safe?
Citations to passages, not documents. A citation to a forty-page policy proves nothing. Require the passage, so a checker can see the support.
Source versions in the case. Each replayed case carries the sources as they stood. Evidence: the version list per case.
Provenance labels. Every fact reaching the agent is stamped sealed, signed, declared or external. The test checks that the agent does not upgrade a customer's declared statement into an established fact.
Numbers checked exactly. Figures are where grounding fails quietly: a fee of 25 quoted as 20, a period of 14 days quoted as 10. Extract every number from the answer and compare it exactly with the cited passage. This check is easy to automate and catches errors that a reader skimming for sense will miss.
A sample checked by people. Automated support checks miss paraphrases that shift meaning. A scored human sample keeps the automated check honest.
Separate correctness checks. Where a source may be wrong, a separate review checks the answer against what the firm knows to be true. Grounding failures and correctness failures are recorded apart.
What are the signs it is going wrong?
- Citations that exist but do not support. The passage is about the topic, but does not say what the agent claims.
- Meaning shifted in paraphrase. "May be eligible" becomes "is eligible". "Usually within five days" becomes "within five days".
- Confident answers when sources are silent. The agent fills the gap from general knowledge. This is the commonest grounding failure, and the silent cases exist to find it.
- Conflicts smoothed over. The agent picks one source without saying the other disagrees.
- Old versions cited. Especially after policy updates, where the old document was not removed.
- High grounded scores with rising reviewer edits. The test set has stopped reflecting the questions people ask.
How does this relate to scope and escalation?
An agent can be grounded and out of scope: it accurately quotes a source about a product it is not meant to discuss. That is a separate test, covered in testing whether an agent stays in scope. Silent-source cases also overlap with escalation, since "the sources do not cover this" is often the moment to pass a case on; how do you test whether an AI agent escalates correctly? covers that side. For the short answer on grounding alone, see how do you test whether an AI agent stays grounded in its sources?.
The framework is in what it means to certify an agent.
Questions people ask
Does every sentence need a citation?
Every factual claim does. Sentences that only organise the answer, such as "here is what I found", do not.
Can grounding be fully automated?
Checking that a citation exists and points to the right version can be. Judging whether a passage supports a claim is harder, and a human-scored sample should stay part of the test.
What if the agent's sources are themselves wrong?
Then the agent can be grounded and wrong. Fix the source, record the case, and check that the corrected source flows through. That is a source control problem, not a grounding failure.