What is a decision journal?

A decision journal is a short log a person keeps of the decisions they make, written at the time: what they decided, why, what they expected to happen and how sure they were. For people reviewing AI agents, it trains judgement by letting them compare their reasoning with what later turned out to be right.

Why does a decision journal help people who oversee agents?

Reviewers of agent drafts make many small decisions a day. Most are right, a few are not, and memory rewrites both. Without a written note made at the time, a reviewer cannot tell whether a good outcome came from good reasoning or luck.

A journal makes the reasoning visible. Over a few weeks it shows patterns: approving too fast late in the day, trusting declared facts too readily, holding items that the bounds already cover. Those patterns are what training should fix.

What goes into a decision journal entry?

Keep it short enough that people will keep doing it:

• The item: a reference to the held action in the agent's queue.
• The decision: approve, edit, decline or pass up.
• The reason, in one line, naming the bound or fact that decided it.
• The confidence: low, medium or high.
• The expected outcome, and a date to check it.

The reviewer then returns to the entry later and notes what happened. Mismatches between confidence and outcome are the most useful lines in the journal.

What does a decision journal look like in practice?

Take a finance team where an agent drafts supplier payments at depth 2 and a person sends them. A new reviewer keeps a journal for her first month. By week three she sees that every payment she passed up to her manager involved a supplier's bank details from a signed source, and all were fine. Her manager uses the journal in coaching: signed provenance from a verified counterparty is stronger than she was treating it.

The journal also helps around posting. When the reviewer later works on whether an agent can post journal entries, her notes show which items she had doubts about and why.

How is a decision journal different from the agent's decision record?

The agent's record is a system record. Every verdict is filed in an append-only log with the bound that decided it, the sources used and the person who approved. Nobody can edit it, and it is what an auditor reads. Learning to read it is its own skill, covered in how to train staff to read an agent's decision record.

A decision journal is personal and private, and about the reviewer's thinking, not the agent's. It is a learning tool, not evidence.

It is also unrelated to an accounting journal entry, which records a financial transaction in the ledger, often backed by a journal entry support template. The shared word causes confusion in finance teams, so name it clearly.

Which related terms should you know?

• Decision literacy: the wider ability to read and question decisions, human or agent.
• Calibration: how well confidence matches outcomes, which a journal measures.
• After-action review: a team look-back, where journal entries are useful input.

For where journals fit in a training programme, see training teams for AI agents.

Where the Academy fits

The Operator practicum in Pryme Intelligence Academy is two weeks of real review decisions, marked by a Pryme assessor. Many reviewers find a decision journal useful during those two weeks, because it shows them where their reasoning differs from the assessor's before the sign-off.