A counterexample is a single case that shows a general statement is false. In training people to oversee AI agents, it is an item that looks like the usual pattern but needs a different decision, such as a refund that matches every routine feature yet must be held because its source is external.
Why do counterexamples matter for people who review agents?
Reviewers learn patterns fast. After a few hundred held drafts, they start to approve on recognition: same customer type, same amount band, same wording. That is efficient, and it is exactly where a bad item slips through.
A counterexample breaks the pattern on purpose. It teaches the reviewer to check the bound that decided, not the surface of the draft. One well-chosen counterexample in a course does more for judgement than ten more examples of the normal case.
How are counterexamples used in practice?
A trainer starts with the rule as people tend to state it, then finds the case where it fails. Some examples:
| Rule people assume | Counterexample |
|---|---|
| "Small refunds are always fine to approve" | A refund of 50 to an account changed yesterday by an emailed request |
| "If the agent drafted it, the policy allowed it" | A draft built on a policy page that was replaced last week |
| "Refused means the agent is broken" | A refusal because the item sits outside the agent's role scope, which is working as designed |
| "Anything the system can do, an agent can be given" | A withheld action, refused at the platform floor for every agent at every depth |
Each counterexample comes with the question the reviewer should have asked, and where in the record the answer sits.
What does a counterexample look like in a real team?
Take a lettings agency whose agent drafts deposit returns at depth 2. New reviewers learn that returns with a clean check-out report are routine. The counterexample: a clean report, a routine amount, but a tenant's bank details supplied by a forwarded email. The provenance is external, and an external claim cannot authorise a payment.
A trainer on training the lettings team to review agent decisions would put this case in the first week, not the last. Sectors with high stakes, such as payment companies and hospitals, need counterexamples in the assessment too, not just the lessons.
How is a counterexample different from an edge case?
An edge case sits at the boundary of what the rule covers: the amount exactly at the limit, the date on the last day. A counterexample sits inside what people think the rule covers and proves them wrong. Edge cases test precision. Counterexamples test whether the reviewer's mental model is right at all.
Both belong in training, but counterexamples are the ones that change behaviour.
Which related terms should you know?
- Edge case: a case at the limit of a rule.
- Mandatory training: where counterexamples are often missing, because courses are built to be passed quickly.
- Training calendar: useful for scheduling refreshers with new counterexamples drawn from recent held items.
For how counterexamples fit into a wider programme, see training teams for AI agents.
Questions people ask
Where do good counterexamples come from?
From your own record. Look for held or refused items that a reviewer nearly approved, and for decisions later reversed. Remove personal details and use them as training cases.
How many counterexamples should a course include?
Enough to cover each bound in the decision chain at least once. A course with no counterexamples teaches recognition, not judgement.