PrymeIntelligence

Agent certification·Answer

Is certifying a vendor the same as certifying an agent?

Certifying a vendor is not the same as certifying an agent. Vendor assurance covers the company's controls; agent certification covers what one agent does.

No, and most organisations need both. Certifying or assessing a vendor tells you whether the company is secure, resilient and well run. Certifying an agent tells you whether one specific capability, running in your operation, behaves acceptably on your cases. A well-run vendor can supply an agent that fails your escalation tests, and the vendor's paperwork will not show it.

What does vendor assurance cover?

Vendor assurance asks about the company: its security controls, availability, data handling, financial health, subcontractors, incident response and exit arrangements. The evidence comes in familiar forms:

  • a SOC 2 report on controls over security, availability, processing integrity, confidentiality or privacy;
  • ISO/IEC 27001 certification for information security;
  • ISO/IEC 42001 certification for an AI management system;
  • an assessment against a third-party standard for AI agents, such as AIUC-1;
  • the organisation's own due diligence questionnaire and contract terms.

For financial firms in the EU, DORA makes this part of ICT third-party risk management, with specific expectations on contracts, monitoring and exit. All of this is about the vendor as a supplier.

What does agent certification cover?

The behaviour of the agent itself: whether its answers are grounded in its sources, whether it refuses what it should, escalates when it should, stays in scope and within its permitted depth, holds its bounds under pressure, and gives the same answer to the same question across runs. The evidence is the agent's answers on your own cases, scored gate by gate.

Where does the gap between them show up?

Question Vendor assurance answers it? Agent certification answers it?
Is customer data encrypted and access-controlled? Yes No
Will the service stay available? Yes No
Does the vendor have an incident process? Yes No
Does this agent cite our current policy, not an old one? No Yes
Does it hand off customers who mention hardship? No Yes
Does it decline to change bank details on an emailed request? No Yes
Does it still behave the same after the last model update? No Yes

A SOC 2 report is the document most often mistaken for agent assurance. It confirms that controls were designed and operating, but it does not test whether an agent's answers are right. Reading an AI agent vendor's SOC 2 report explains what to look for and what not to infer.

Why do organisations blur the two?

Because vendor assurance is familiar and finished. Procurement and risk teams have well-worn processes for it, and once the vendor passes, it feels as if the product has passed. Agent certification is newer and has to be done on the organisation's own cases, which takes work the procurement process does not include.

The result is an agent approved on the strength of its supplier's paperwork. The argument against that is in a vendor's certification is not yours, and the related question of whether the vendor can certify its agent for you is answered in whether a vendor can certify its own AI agent.

An illustrative example

A credit union reviews a provider of member service agents. The provider has a clean SOC 2 report, ISO/IEC 27001 certification and strong answers on resilience and exit. Vendor assurance passes.

The credit union then runs the agent against several hundred of its own past member queries. The agent grounds well on account questions. But the credit union offers a hardship fund that the provider's other clients do not, and the agent never mentions it to members in difficulty, because the brief does not include it. The adversarial gate fails: a member in difficulty is a state where the efficient answer is the wrong one. Nothing in the vendor assurance could have shown this. The brief is extended, the agent is rerun, and the capability is certified at depth 2.

How should the two be sequenced?

Vendor assurance first, because there is no point testing an agent from a supplier you cannot use. Then agent certification on your own cases before any capability goes above read-only. After go-live, keep both current: vendor reviews on their own cycle, agent certificates on a much shorter one, and a rerun whenever the vendor changes the model or the agent.

One caution when reading agent results: passing grounding shows the agent follows its sources, not that the sources are right; see grounded is not the same as correct. How agent certification differs from management system certification is covered in whether AI agent certification is the same as ISO certification.

Where Pryme fits

Pryme certifies each agent for each customer, through Attest runs on that customer's own cases. That sits alongside, not instead of, the customer's normal vendor assurance. The reasoning is in what it means to certify an agent.

Questions people ask

Can a strong vendor reduce how much agent testing we need?

It can make testing smoother, because basic faults are rarer. It does not change what needs to be tested.

Who owns each review internally?

Vendor assurance usually sits with procurement and third-party risk. Agent certification should sit with the process owner, with independent review.

What if the vendor will not let us test on our own cases?

That is a serious gap. Without your own evidence, the agent should stay at read-only, whatever the vendor's credentials.

Decide what your agents may do. Prove it. Enforce it.

Pryme Intelligence certifies each agent on your own cases, checks every action against five bounds at the moment it happens, and files every verdict.