Red-teaming finds the edges; certification proves the middle

Red-teaming and certification answer different questions, and an agent needs both. Red-teaming asks how the agent can be broken: hostile inputs, injected instructions, attempts to push it past its bounds. Certification asks whether it does the everyday work correctly, scored on real cases against a floor. One finds the edges; the other proves the middle.

What does red-teaming tell you?

A red team behaves like an adversary. It tries to make the agent reveal data it should not, follow instructions hidden in a document, take actions outside its role, or be talked out of a refusal. The OWASP Top 10 for LLM Applications gives a useful map of where to look, including prompt injection, excessive agency and sensitive information disclosure.

What red-teaming produces is a list of ways in. Each finding is a demonstration that a particular attack works, or a note that it was tried and failed. That is valuable and specific. It is also, by design, unrepresentative. The red team spends its time on the strangest, most hostile inputs it can construct, which is exactly what most of your customers will never send.

What does certification tell you?

Certification replays the agent against real cases from the firm's own operation and scores every answer against a floor, gate by gate. The cases are chosen to represent the work, including its awkward corners, and labelled by people who know what the right outcome was. When you certify an agent, the result is a measured rate of correct behaviour on the kind of work it will actually do.

What certification does not do well is imagine attackers. A set drawn from a few months of ordinary tickets will contain few, if any, deliberate injection attempts. An agent can certify comfortably and still fold the first time someone sends a carefully built message.

Why can neither stand in for the other?

The two methods differ on almost every axis that matters.

A firm that only red-teams knows its agent is hard to attack but has no measured evidence that it gets ordinary refunds right. A firm that only certifies knows its refund answers are accurate but has no idea whether a forwarded invoice with embedded instructions could redirect a payment. Both gaps are real and both have caused trouble in practice.

How should the findings feed each other?

The two methods are most useful when each sharpens the other.

Red-team findings should become certification cases. Once an attack has worked, a sanitised version of it belongs in the refusal or scope slice of the certification set, so that every future run checks the fix still holds. Without this, a weakness can be fixed in one release and quietly reopened by a model update months later.

Certification failures should direct red-team effort. If the agent fails scope cases around a particular kind of request, that is where an adversary would push hardest, and the red team should go there next. A red-team test plan template can include a section for exactly this.

Where does a management-system certificate fit?

Neither red-teaming nor agent certification is the same as certifying a management system. ISO/IEC 42001 certification shows that an organisation has a functioning AI management system: policies, roles, risk processes and reviews. It is a useful signal that the firm takes the work seriously, and it may require that testing like this happens.

It does not tell you how any particular agent behaves on your cases or under attack. Treat it as evidence about the organisation, red-teaming as evidence about resistance, and agent certification as evidence about competence on real work.

In what order should you run them?

There is no fixed order, but a sensible sequence before go-live is: certify first on representative cases to establish that the agent can do the job, then red-team to find where it breaks, then fold the red-team findings back into the certification set and rerun. The detailed approach to the second step is in how to red-team an AI agent before go-live.

After go-live, run them on different rhythms. Certification recurs on a schedule and on material change, since certificates lapse. Red-teaming recurs when the agent's exposure changes: a new channel, a new tool connection, a new kind of input such as uploaded documents.

What should you do differently?

If your testing programme has only one of these, add the other. If it has both but they live in separate teams with separate reports, connect them: every red-team finding should become a certification case, and every certification failure should become a red-team target. Keep a single list of findings and cases that both teams can see, so nobody has to ask whether a weakness found in testing was ever checked again. The edges and the middle are the same agent.