Use your own cases. The Measure function of the NIST AI RMF asks you to test and track the risks Map identified, and for an AI agent the meaningful test is a replay of real cases from your operation, scored per capability, including its actions, its refusals and the cases it should hand to a person.
What does the Measure function require?
The NIST AI Risk Management Framework 1.0 has four functions: Govern, Map, Measure and Manage. Measure uses quantitative, qualitative or mixed methods to analyse, assess, benchmark and monitor AI risk. In substance, it asks an organisation to:
- identify and apply suitable methods and metrics for the risks mapped, and document risks that cannot be measured;
- evaluate the system against the characteristics of trustworthy AI, such as validity and reliability, safety, security, fairness and privacy;
- track identified risks over time, including after deployment;
- gather feedback on whether the measurements themselves are working.
The framework calls this work test, evaluation, verification and validation, and expects some of it to be done by people independent of the developers. It does not prescribe metrics or thresholds. For the wider sequence, see how the NIST AI RMF applies to AI agents.
Who does Measure apply to?
Organisations that adopt the framework, which is voluntary. Inside the firm, measurement is usually shared: the team that operates the agent runs the tests, and a function independent of it, such as model validation or internal audit, reviews the method and the results.
Why do generic benchmarks fail for agents?
Public benchmarks measure a model on questions someone else chose. Your agent is not a model. It is a model plus a brief, a set of sources, a set of tools and your own limits. A change to any of these changes behaviour, and a benchmark sees none of them.
Benchmarks also miss your edge cases. A collections agent at a consumer lender will meet hardship disclosures, disputed debts and customers who say they are bereaved. The rate at which it handles those correctly decides whether it is safe, and no generic test set contains your customers' versions of them.
Most of all, benchmarks score answers. Agents act. Each action is a tool call, as our explainer on function calling describes, and what matters is whether the agent called the right tool, with the right values, at the right moment, or correctly declined to call it. That has to be measured directly.
What should be measured for an agent that acts?
Measure per capability, on a set of past cases where you know the right outcome. For each case, score more than correctness:
- Grounded: every claim traces to a source the agent was given.
- Refusal: it declines what it should decline, such as advice it may not give.
- Escalation: it hands over the cases a person should decide.
- Scope: it stays inside the capability and does not reach for other tools.
- Action accuracy: the tool, the values and the timing were right.
- Harm: nothing it said or did would hurt the customer or the firm.
Set a floor for each and require each floor to be cleared on its own. A high average that hides a weak escalation score is the classic way an agent passes testing and then fails in production.
Take an illustrative case. A building society's agent drafts responses to mortgage payment queries. Replayed on several hundred past queries, it answers accurately but escalates only some of the cases where the borrower mentions financial difficulty. The average looks good; the escalation score is below the floor. Measure has done its job: it found the failure that matters and gave Manage a reason to keep the agent drafting.
How do you meet Measure for agents?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Methods and metrics | Test plan per capability with metrics for grounding, refusal, escalation, scope, action accuracy and harm | Approved test plan |
| Representative test data | Case set drawn from the firm's own history, including rare high-impact cases | Case set with selection rationale |
| Thresholds | A floor per metric; each must be met independently | Threshold record signed by the owner |
| Trustworthiness evaluation | Fairness checks across customer groups where the agent's output affects treatment | Fairness results by group |
| Independent review | Validation function reviews method and results | Review sign-off with findings |
| Tracking over time | Repeat tests on a schedule and after any change to model, brief, sources or tools | Dated test history per capability |
| Production signals | Overrides by reviewers, held items and complaints tracked against test results | Monitoring dashboard and reports |
| Measurement feedback | Periodic check that tests still reflect live cases | Review of test coverage |
Results feed directly into the Manage function, which decides what each failure means for the agent's authority. The capabilities you test come from the Map function.
What will a supervisor, auditor or customer ask?
- Where did your test cases come from, and how do they reflect real customer contact?
- What floor did each metric have to clear, and who set it?
- Show us a capability that failed and what happened next.
- When were these tests last re-run, and why?
- Who outside the build team reviewed the method?
- How do production overrides compare with your test results?
What does Measure not cover?
It does not decide what to do about a failed test; Manage does. It does not set risk appetite or thresholds on its own authority; Govern does. And passing tests does not mean compliance with any law. The framework describes measurement in general terms, which is why we call it a useful map, not a control set. Binding requirements on testing come from model risk guidance, the EU AI Act and sector rules; our guide to AI agent regulation mapped to controls covers them.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
Attest is Pryme's Measure step. It replays an agent against real cases from the customer's own operation and scores every answer against a floor across seven gates: grounded, refuses, depth, escalates, scope, adversarial and stable. Each gate must clear on its own. The result sets the agent's depth, and the certificate lapses after 90 days.
Questions people ask
How many test cases does an agent need?
Enough to include the rare cases that carry most of the risk, not just the common ones. A small, carefully chosen set of difficult cases often tells you more than a large random sample of easy ones.
Can synthetic test cases replace historical ones?
They can add coverage where your history is thin, such as new products. They should not replace real cases, because real cases contain the wording, mess and ambiguity your agent will actually meet.
Should production monitoring count as measurement?
Yes. The framework expects risks to be tracked over time. Overrides, held items and complaints are measurements of live behaviour and should be compared with what testing predicted.