Credit analysts should drill a data failure that silently changes decisions: the bureau connection starts returning another applicant's file, and the agent's credit summaries look normal. Over about 90 minutes the credit team practises pausing the agent to draft-only, escalating to the chief credit officer and data protection, recording each decision, reviewing affected applications and reporting.
What does a credit team drill rehearse?
Credit analysts assess applications for loans, cards or business facilities. Agents typically gather documents, pull bureau data, draft a credit summary against policy, and propose a decision at depth 2 for an analyst to approve. Some lenders let agents request missing documents at depth 3 within bounds. Credit decisions about individuals are sensitive: the EU AI Act lists creditworthiness assessment of natural persons as high-risk, and in the US, ECOA and Regulation B require specific reasons when credit is denied.
The chief credit officer owns the drill. A senior analyst or credit risk manager facilitates. Analysts, someone from credit risk, the data protection officer and an engineer who knows the bureau connection take part. For the format, see what an incident drill is.
The scenario
Wednesday, 15:00. An analyst reviewing a credit summary notices the bureau section lists a mortgage the applicant says they have never had. Checking a second file, she sees the same mortgage. An engineer confirms that a change to the bureau connection on Monday mismatched requests and responses under load. In two days, 250 summaries were drafted; analysts approved 140 decisions, including 35 declines.
The drill in nine steps
- Person (facilitator): reads the scenario and starts the clock.
- Person (analyst): escalates after the second file, not after the tenth. The drill rewards the instinct to stop at the pattern.
- Person (credit team lead): pauses the agent to draft-only. Document requests that ran at depth 3 are held too. Every verdict now names the lender's own limits.
- Person (credit team lead): escalates to the chief credit officer, the engineer and the data protection officer.
- Person (engineer): confirms the window when responses were mismatched and fixes or disables the connection.
- Person (analyst pairs): list affected applications from the record and re-pull bureau data for each.
- Person (chief credit officer): decides on remedy. Reassess every decision made in the window. Write to declined applicants whose outcome changes. Hold any disbursement not yet made.
- Person (data protection officer): decides on data breach reporting. Applicants' credit files were shown to the wrong people, which may need notification under data protection rules.
- Person (credit team lead): records each decision; the facilitator runs the debrief.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Alert | Has drafted summaries | 2 | Spotting the pattern |
| Pause | Holds every action | Dropped to 2 | Tightening limits |
| Escalate | Nothing new | 2 | CCO, engineer, DPO |
| Fix connection | Nothing | 2 | Engineering |
| Scope and re-pull | Redrafts summaries on fresh data | 2 | Checking each one |
| Remedy | Drafts letters | 2 | Reassessed decisions, holds |
| Report | Nothing | 2 | Data breach and regulator decisions |
Why is this scenario hard for credit analysts?
The bureau data arrived through a verified connection, so it carried a signed label: an identifiable party stood behind it. Analysts are trained to trust signed data more than declared data from the applicant. In this case the applicant's own statement, that they had no mortgage, was right, and the signed source was wrong.
The lesson is not to distrust signed data. It is that a contradiction between a signed source and a declared one is a reason to stop and check, not to side with the stronger label automatically.
Which controls does the drill test?
The pause. Evidence: the limit change in the record.
Scope from the record. Every summary and every approval is filed append-only. Evidence: the list of affected applications and the analyst who checked each.
Reasons for reassessed declines. Evidence: new reasons signed by a named analyst, filed with the fresh data.
Breach decision. Evidence: the data protection officer's written decision and any notification.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
What should the chief credit officer measure?
- Files reviewed before the first escalation. Two is good; ten means the team needs work.
- Time from the alert to the pause.
- Time to reassess every decision in the window.
- Whether every changed decline produced a new, specific reason.
Where do credit drills go wrong?
Analysts correct the file and move on. One wrong bureau section looks like a one-off. The drill should make the second-file instinct automatic.
Reassessing only declines. Approvals made on someone else's clean file may be wrong too.
Data protection left until last. Breach rules often have short deadlines. Involve the DPO at the escalation step.
Credit teams sit between neighbours with related risks: the credit control team drill covers collecting what was lent, and the IT team drill covers the technical response to a broken connection. The AI agent incident drill script template gives a ready structure, and how to train a team to handle an AI agent incident covers training. The full approach is in training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy's Operator track has a module on how a governed agent decides, including provenance classes, and another on when something goes wrong. Its written exam is 40 questions in 60 minutes, passed with at least 32 correct, and the practicum requires no missed red line across two weeks of real review decisions.
Questions people ask
Should the agent pull bureau data itself?
Yes, at depth 1 through a verified connection. The analyst should still compare the bureau data with the applicant's own statements and stop when they disagree.
Do we need to tell applicants their data was shown to someone else?
That depends on the facts and the data protection rules that apply. The drill should practise the decision, made by the data protection officer, so the real one is quick.
How often should credit teams drill?
At least twice a year, and after any change to data connections or credit policy.