Assess the customer service and returns team before the first agent touches orders, and do it well before peak trading. The test, set in retail, covers refunds and depth, evidence from customers, held actions and the record. The report shows the gaps by concept and decides whether the first agent answers order questions, authorises returns or drafts refunds.
What is the assessment for in an online retailer?
Online retailers handle huge volumes of near-identical contacts: where is my order, can I return this, my item arrived damaged. Agents can answer many of these at depth 3 within policy, authorise returns, and draft refunds. A refund moves money, which is depth 4 work, so most retailers keep refunds above a small amount held for a person.
The people who review those holds decide quickly, in large numbers, often during the busiest weeks of the year. The assessment checks whether they understand what they are approving. The head of customer service usually owns it, with the returns operations lead. For the concept, see what a readiness assessment is.
Who should take it?
One team of up to 25 from the channel where the first agent will work:
- Customer service advisers, including seasonal staff if the first agent will run through peak.
- Returns and refunds staff who approve held refunds.
- A fraud or loss prevention analyst, who will see refund abuse patterns the advisers do not.
- Team leaders who handle complaints and escalations.
Marketplace and seller support teams, if the retailer runs one, take their own sitting; their agents deal with sellers, not shoppers. The order of training is covered in who in an online retailer needs AI agent training first.
The steps
- Person (head of customer service): chooses the team and date, at least ten weeks before peak trading begins.
- Person (head of customer service): asks for the test set in retail, with questions on orders, returns and refunds.
- Person (each member): sits the test alone.
- Person (head of customer service and fraud analyst): read the report by concept.
- Person (head of customer service): chooses the first agent and its depth. A common start is order status and delivery questions at depth 3, with returns drafted at depth 2.
- Person (team leaders): train the weakest concept before the trading freeze.
- Agent, depth 1: reads last season's closed contacts in shadow and proposes the reply and action it would have taken.
- Person (cohort): reassesses before the agent drafts live replies.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Choose team and date | Nothing yet | None | Timing against peak |
| Sit the test | Nothing yet | None | Individual answers |
| Read the report | Nothing yet | None | Gaps, with fraud |
| First agent | Named as candidate | None | Agent and depth |
| Train the gap | Runs in practice labs | Practice | Learning |
| Shadow | Proposes replies on past contacts | 1 | Comparing with what was done |
| Reassess | Nothing new | 1 then 2 or 3 | Go-live |
What does the test usually find in online retail?
Evidence for refunds. A courier's delivery scan is signed by the carrier. A customer's statement that the parcel never arrived is declared. A photo of a damaged item sent by the customer is external. Advisers trained to "keep the customer happy" often treat all three as enough to refund. The test shows whether they read the agent's provenance stamps, and whether they know that an external claim cannot authorise a payment on its own.
Depth for refunds. Many staff think of refunds as routine. In governed terms they move money, which is depth 4, and the retailer's own limits decide which amounts the agent may refund without a person. Advisers who misunderstand this will expect the agent to refund anything a customer asks for, and will treat every hold as a fault.
The record during peak. When volumes spike, people approve faster. Staff should know that every approval names them and can be reconstructed if a refund pattern turns out to be abuse.
What does the report change?
It sets the first capability. A team weak on refund evidence should start with order status answers, which carry no money, and leave refunds with people until after peak.
It sets the peak plan. If seasonal staff will approve agent drafts, they need the weak concept in their induction, and the retailer may decide to keep the agent at depth 2 through peak.
It sets the retailer's own limits. If the fraud analyst sees that advisers approve refunds on external evidence, the refund limit for the agent and for new advisers can be set lower.
What evidence should the retailer keep?
The report by concept.
The decision on the first agent, its depth and the refund limit.
The peak plan, naming which concepts seasonal staff are trained on.
The reassessment before live replies.
What should the head of customer service measure?
- Scores by concept, comparing permanent and seasonal staff.
- During shadow, contacts where the agent would have refunded and the team did not, or the reverse.
- After go-live, refunds approved on external evidence alone, found by sampling.
Where does it go wrong in online retail?
Running it in the trading freeze. No time to train, no time to reassess.
Excluding loss prevention. The fraud analyst is the person most likely to spot what the test reveals about refund habits.
Choosing refunds as the first agent because they cost the most to handle. They cost the most to get wrong, too.
A wider plan follows in how to roll out AI agent training across an online retailer, and directors' needs are covered in what AI agent training an online retailer board needs. The hospital version is a useful contrast where held messages carry safety rather than money. The topic guide is training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy's free readiness assessment covers teams of up to 25, with the Foundations test set in the team's own industry and a report showing the gaps and the first agent to start with. Lessons, labs and tests are free, so seasonal staff can work through the material before peak without a cost per head.
Questions people ask
Should seasonal staff take the assessment?
If they will approve anything the agent drafts, yes, as part of induction. If they only handle contacts the agent never touches, it can wait.
Can the agent issue refunds on its own?
Within the retailer's own limits and the capabilities it is certified for, yes. Most retailers start with small amounts and raise them on evidence.
How close to peak is too close?
Allow at least ten weeks: time to assess, train, shadow and reassess before the freeze.