Learning analytics is the collection and analysis of data about learners and their learning, used to improve courses and spot who needs help. For training on AI agent oversight, the most useful learning data is not course completion but how people decide held actions afterwards, which the agent's record already captures.
Why does learning analytics matter for agent oversight?
Most learning data answers the wrong question. Completion rates, time spent in a module and quiz averages tell you whether people went through the training. They do not tell you whether reviewers now catch the payment with an external bank detail.
Agent oversight gives you something better. Every verdict and every approval is filed, with the reviewer's name and reason. Joining that record to training data shows which lessons change decisions and which do not.
What does learning analytics measure for agent reviewers?
Three layers, from easiest to most useful:
| Layer | Example measures | What it tells you |
|---|---|---|
| Activity | Enrolment, completion, time in lessons | Whether people took the training |
| Assessment | Scores by concept, items most often missed | What they understood at the time |
| Work | Reasons recorded on approvals, held items passed up, approvals later reversed | Whether training shows in real decisions |
The work layer needs care. It should be used to improve training and target coaching, not to rank individuals publicly.
What does learning analytics look like in practice?
Take a payroll bureau that trained 40 reviewers on an agent that drafts pay variations. Assessment data shows everyone scored well on depth and verdicts, but many missed items on provenance. The work data, a month later, shows the same pattern: approvals of variations based on an employee's emailed request, with no check against the signed contract.
The bureau rewrites the provenance lesson, adds two lab exercises, and checks the work data again the following month. Without the work layer, the first course would have looked like a success.
How is learning analytics different from a learning record?
A learning record is the individual's history: what they took, when, and what they scored. Learning analytics works across many records, looking for patterns and linking them to outcomes. The record is the raw data; analytics is what you do with it.
Learning analytics also differs from performance management. Analytics asks whether the training works. Performance management asks whether the person does. Mixing the two makes people defensive about the data.
Which related terms should you know?
- Learning path: the route through training for a role, which analytics can show is too long or missing a step. See how to create a learning path for agent roles and the learning path template for agent roles.
- Blended learning: a mix of formats, where analytics shows which format teaches which concept best.
- Training evaluation: judging whether a programme worked, for which analytics supplies the evidence.
The role L&D plays in turning this data into action is covered in the role of learning and development in agent rollouts. For the overall approach, see training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy gives team admins a view of progress, results, certificates and expiry dates. For teams of up to 25, the free readiness assessment reports Foundations scores by concept and shows the gaps, which is the assessment layer of learning analytics from day one.
Questions people ask
Is it fair to use review decisions as learning data?
Yes, if people know it is used, and it is used to improve training and coaching rather than to discipline. Agree the rules before collecting.
What is the single most useful learning metric for reviewers?
Approvals later reversed, broken down by concept. It points directly at what training missed.