What is work-based assessment?

Work-based assessment judges a person's competence using evidence from their real work, done in their real role, under normal conditions. Instead of an exam room or a simulation, the evidence is what they actually did: decisions made, cases handled, outputs approved. The assessor looks at the work and asks whether it meets the standard.

Why does it suit AI agent oversight?

Reviewing an agent's work is done in a queue, under time pressure, with incomplete information and a steady stream of items that look alike. Those conditions are hard to reproduce in a test. A person who answers every exam question correctly may still approve an unverified bank change at the end of a long shift. Work-based assessment sees that; an exam does not.

It also produces evidence others trust. A client or auditor asking how you know a reviewer is competent will give more weight to a signed-off sample of real decisions than to a quiz score.

How does it work in practice?

The design questions are about sampling and safety:

• What to sample: all decisions for a period, or a sample weighted towards higher-risk items.
• How to judge: against a written standard, with red lines that fail the assessment on their own.
• Who judges: a qualified assessor, ideally outside the learner's reporting line.
• How to keep it safe: while the learner's decisions are live, a second person checks anything that cannot be undone.
• What to record: the sample, each judgement, the red lines, the outcome and the assessor's name.

The sampling choice matters most. An AML team assessed only on routine alerts will look competent until the first complex one arrives. Weighting the sample by risk follows the same logic as the FATF risk-based approach; see training AML staff who work with agents.

What is an example?

A procurement team adopting an agent that drafts purchase orders assesses each buyer on two weeks of real approvals. The assessor checks every approval over a set value and a sample of the rest. One buyer is accurate throughout but approved two orders to a supplier added from an emailed quote without checking the supplier record. The standard says supplier records must be verified before first payment. The assessor sets a further week of assessed work focused on new suppliers, then signs off. A test would never have shown the habit.

How is it different from competency-based assessment?

Competency-based assessment is defined by what is judged: specific competencies against a standard. Work-based assessment is defined by where the evidence comes from: real work. They are often combined, and the strongest assessments for reviewers are both. A competency like "never approves on external evidence alone" is best shown on real work.

It also differs from an AI risk assessment, which assesses a system, and from a needs assessment, which finds gaps before training; the AI literacy needs assessment template covers that step.

Which terms are related?

• Practicum: a defined period of work-based assessment before certification.
• Assessor: the person who judges the work.
• Evidence of competence: what the assessment produces.

For the wider topic, read AI literacy and certification for people who work with agents.

Where the Academy fits

The Certified Pryme Operator practicum in Pryme Intelligence Academy is work-based: two weeks of real review decisions, at least 95 in every 100 correct, no missed red line, signed off by a Pryme assessor. It follows the written exam, so knowledge and real work are both evidenced.