A shadow period is the stretch before go-live when an AI agent works on real cases alongside people, without its actions taking effect, while people keep doing the work. Most organisations treat it as a technical check to tick off. It is the best training opportunity the team will have: real cases, side by side, with nothing at risk.
What is a shadow period, and what is it not?
During a shadow period, the agent receives the same live cases people handle. It reads, drafts and proposes as it would in production, and its verdicts are recorded, but nothing it proposes is sent, executed or paid. People carry on as normal. Afterwards, the agent's output is compared with what people actually did. The definition is set out in what a shadow period is.
It should not be confused with shadow AI, which is staff using unapproved AI tools outside governance. The two share a word and nothing else. What shadow AI is explains that separate problem, and how to stop shadow AI agents covers the controls. A well-run shadow period, oddly, is one of the best defences against shadow AI, because staff who have seen a governed agent up close are less tempted by ungoverned ones.
Why do organisations treat it as a formality?
Because the agent's builders own it. The shadow period is often run by whoever configured the agent, to check that it behaves as expected before certification or go-live. They compare outputs, log mismatches and report a match rate. The people who will review the agent's work in production may not even know it is happening.
That wastes most of its value. The comparison between what the agent proposed and what a person did is exactly the material reviewers need to learn from. Each mismatch is a question: was the agent wrong, was the person wrong, or was the case ambiguous? Answering it teaches reviewers how the agent thinks and teaches the organisation where its policy is unclear.
What should reviewers learn during a shadow period?
Four things that are hard to learn any other way:
- How the agent reasons on familiar cases. Reviewers see drafts on cases they handled themselves that day, which makes the agent's logic easy to follow and its errors easy to spot.
- Which bound decides most verdicts. Every action meets five bounds at once, and the lowest wins. Seeing which bound held or refused most items shows where the organisation's own limits sit in practice.
- How provenance changes the picture. Reviewers see the same facts they used, labelled sealed, signed, declared or external, and notice when they themselves trusted something they should not have.
- What their own decisions look like on the record. Every verdict is filed in an append-only record. Comparing it with their own handling shows reviewers what an auditor will later see.
How do you run a shadow period as training?
Put reviewers at the centre of it. Daily, have them look at a sample of cases where the agent's proposal differed from what a person did. They decide which was right and why, and write it down. Weekly, bring the team together on the hardest mismatches and agree the standard. Feed findings into the agent's brief, sources and certification cases, and into the training plan.
Record the results in a structured way so they can be used after go-live; the shadow run report template gives a format. The report should cover the agent and the people: where the agent erred, where people erred, and which concepts reviewers found hardest. That last section becomes the first draft of the training plan for go-live.
What is the trade-off?
Time from the people who are already doing the work. A shadow period run as training asks reviewers to do their normal job and also review the agent's proposals. That is extra effort for weeks, and teams already busy will resist it.
The cost is real but bounded. A few hours a week for a few weeks, against a go-live where reviewers already understand the agent, know where it is weak and have agreed standards for the hard cases. Organisations that skip this tend to discover in the first month of production what they could have learned with nothing at risk. A mismatch found in shadow costs a conversation; the same mismatch found live may cost a customer, a payment or a complaint. Some shadow periods also test data handling, such as how an agent deals with records whose retention period has ended; whether an AI agent can delete data when retention periods end is the kind of question better answered in shadow than live.
What should you do differently?
Give the shadow period to the reviewers, not only the builders. Schedule daily mismatch reviews and weekly team sessions. Track what reviewers learn as well as what the agent gets right. End the shadow period with a short assessment of reviewers on the concepts they will need, not just a match rate for the agent.
Then carry the habit into go-live. The daily review of differences becomes the daily review of held items, and the weekly session becomes calibration. Pryme Intelligence Academy's labs run in a free Build workspace, which teams can use to practise before their own shadow period begins. The full approach is in training teams for AI agents.
Questions people ask
How long should a shadow period last?
Long enough to see the full range of cases, including month-end or seasonal peaks if they matter. For many processes that means several weeks rather than days.
Should the agent be certified before or after the shadow period?
Shadow evidence supports certification, which replays the agent against real cases and scores every answer against a floor. Many organisations shadow first, certify, then go live at a conservative depth.
Who should own the shadow period?
Jointly, the builder who owns the agent and the operations lead who owns the reviewers. If only one side owns it, either the agent or the people will be left untested.