How to run calibration sessions for loan officers

Loan officers calibrate on the applications an agent held before they reach credit: incomplete files, borderline cases and applicants who need a careful message. Each officer decides the same set alone, including what they would tell the applicant. Compare, agree a gold standard with the lending manager, and update the agent so its document requests and messages follow it.

How is calibrating loan officers different from credit analysts?

Credit analysts decide whether to lend. Loan officers sit between the applicant and that decision. They gather documents, explain what is needed, decide when a file is complete enough to refer, and tell applicants what happens next. An agent in origination typically checks completeness, drafts document requests and applicant updates at depth 2, and prepares the file for credit.

So loan officers calibrate on different questions: is this file ready to refer, what else should we ask for, and how do we say it. Inconsistency shows up as applicants asked for different documents depending on the officer, or files referred half-complete by one officer and held for weeks by another. The general method is in how to run calibration sessions for AI agent reviewers.

The lending manager owns the sessions. A senior loan officer facilitates. A credit analyst attends to say what credit actually needs.

Which applications belong in the set?

Eight to ten applications from the past fortnight, anonymised:

• A self-employed applicant who uploaded three months of statements when policy asks for twelve.
• A broker-submitted application where the broker's summary of income differs from the documents.
• An applicant who mentions a recent bereavement as the reason for missed payments on another account.
• A file complete except for one signature.
• An applicant asking why they need to provide a document they consider private.
• A draft applicant update from the agent that is accurate but full of jargon.

The session, step by step

1. Person (senior loan officer): selects and anonymises applications.
2. Person (each officer): decides each alone: refer to credit, request more, or hold, and writes the message to the applicant or broker.
3. Agent, depth 1: shows its completeness check, its drafted request or update, and the provenance of each document: uploaded documents are external, the broker's summary is declared, data received directly from a known source is signed.
4. Person (facilitator): reveals the spread on decisions and puts two messages side by side.
5. Person (officers and credit analyst): discuss what credit needs and what officers asked for.
6. Person (lending manager): agrees the gold standard.
7. Person (lending manager): sends the standard to the brief owner, including document request templates and message wording.
8. Person (lending manager): samples referred files and applicant messages for a month.

What does a gold standard look like for loan officers?

It tells officers what is enough and how to ask. For example: "Self-employed applicants are asked for twelve months of statements in one request, listing every document needed. No file is referred to credit with fewer than twelve months unless credit has agreed an exception in writing."

On brokers: "Where a broker's income summary differs from the documents by more than a set margin, the documents are used, and the officer tells the broker which figure was used and why."

On vulnerability: "Where an applicant mentions bereavement, illness or hardship, the officer calls rather than writes, and the agent holds every such file for a person."

On language: "Messages to applicants use plain words, say what is needed and by when, and explain why in one sentence."

What does a session look like?

Take a hypothetical team of eight loan officers. On the three-month statements case, three referred the file anyway with a note, three requested nine more months, and two asked for accounts instead. The credit analyst said files referred short were sent back every time, adding a week. The standard became one complete request up front. The brief owner changed the agent's document request template to list every item in one message, and referrals returned by credit dropped over the following month.

Which controls does calibration support, and what evidence does it leave?

Consistent document requests. Evidence: templates and sampled requests.

Complete referrals. Evidence: files returned by credit, before and after.

Fair treatment. Evidence: the vulnerability standard and sampled files.

Clear communication. Evidence: agreed wording and sampled messages.

What should the lending manager measure?

• Agreement on refer, request or hold.
• Files returned by credit as incomplete.
• Number of separate document requests per application, which should fall.
• Complaints about the application process.

Where does it go wrong with loan officers?

No credit analyst in the room. Officers guess what credit needs.

Ignoring broker cases. Broker submissions behave differently and are often where documents and summaries diverge.

Calibrating decisions but not messages. Applicants experience the message, not the decision.

Compliance and procurement officers calibrate on judgement in different settings; see calibration for compliance officers and for procurement officers. For planning, see the AI agent training plan template for loan officers and how long it takes to train loan officers to supervise AI agents. The topic guide is training teams for AI agents.

Where the Academy fits

Loan officers who approve the agent's document requests and messages every day fit Pryme Intelligence Academy's Operator track. Its practicum is two weeks of real review decisions with no missed red line, and for loan officers the usual red line is a vulnerable applicant's file handled without a person.