PrymeIntelligence

Regulated autonomy·Answer

Should AI agents be fully autonomous?

AI agents should not be fully autonomous as a whole. Single capabilities can run alone once proven; irreversible ones keep a person in charge.

No, not as a whole. Full autonomy belongs to a single capability, not to an agent. Some capabilities can run without a person once they are proven on your own cases, can be reversed and sit inside firm limits. An agent with every capability unattended would include actions nobody could reverse, and nobody would be accountable for them.

What would full autonomy actually mean?

A fully autonomous agent decides what to do and does it, with no person checking before or after. Applied to everything the agent can do, that includes reading data, writing records, messaging customers, moving money and making decisions about people.

Put that way, few organisations would choose it. Yet "fully autonomous" is often used loosely to mean "runs on its own most of the time", which hides the question that matters: which actions does it take alone, and which does it hand to someone? For definitions, see what an autonomous agent is and what a semi-autonomous agent is.

Why is it the wrong goal in regulated work?

Three reasons.

Some actions cannot be reversed. Payments out, external messages, filings, data disclosures and decisions people act on all leave your control. If an autonomous agent takes one wrongly, there is nothing to roll back. The argument is set out in why autonomy in regulated industries is a question about reversal.

Accountability needs a person. Regulators in banking, insurance and healthcare expect a named individual to answer for decisions. In the UK, the Senior Managers and Certification Regime makes that explicit for financial firms. An agent cannot be that person, so someone must stand behind what it does, and that person needs a point of control.

Evidence is per task. An agent proven to match invoices accurately has not been proven to approve payments. Full autonomy assumes evidence that only exists for some capabilities.

Where can a capability run without a person?

A capability can run unattended when four conditions hold together:

  1. its effects can be reversed or are small enough to absorb;
  2. it has been tested on your own past cases and passed;
  3. the facts it relies on come from sources you trust;
  4. it sits inside the limits your organisation has set.

Examples that often meet them: routing tickets, matching invoices within tolerance, updating contact details from a verified channel, freezing a card at the customer's request, sending a templated acknowledgement. Each is reversible or protective, bounded and easy to check.

Examples that rarely meet them: declining credit, paying a new supplier, closing an account with a balance, sending a free-text reply that states a decision.

What stays with a person?

  • Every action that cannot be undone or offset.
  • Decisions with legal or similarly significant effects on a person, which GDPR Article 22 and its UK equivalent restrict when based solely on automated processing.
  • Exceptions, conflicts and anything the agent marks as uncertain.
  • Setting and changing the agent's authority.
  • Reviewing the record of what the agent did alone.

That last item is what makes unattended capabilities safe to grant. A person does not approve each action, but a person reads the pattern and can lower the agent's authority the same day.

What should you aim for instead?

Aim for the right autonomy per capability, earned on evidence and enforced at the moment of action. Some capabilities will reach unattended execution quickly. Others will stay at draft permanently, because the action is irreversible. Both are correct outcomes.

This is a more useful target than "full autonomy" because it can be measured. You can say exactly which capabilities run alone, what evidence supports each, and what happens when one fails. For how to set the level, see how autonomous an AI agent should be. For the conditions an agent must meet before any capability runs alone, see what a fully autonomous AI agent needs before it can be trusted.

Where Pryme fits

In Pryme, there is no switch for full autonomy. Each capability earns its own depth from Attest evidence on the customer's own cases, under a ceiling the customer sets, and the lower of the two wins. Some actions are withheld for every agent at every depth, such as reversing a settled payment on our own rail. Certificates lapse after 90 days, so unattended authority has to be re-earned.

Questions people ask

Will agents become fully autonomous as models improve?

Better models will let more capabilities pass their tests, so more will run alone. Irreversible actions will still need a person, because a better model makes fewer mistakes, not none, and an irreversible mistake costs the same however rare it is.

Is a person approving every action too slow?

Yes, which is why reversible, bounded capabilities should run alone. Keep approval for actions where it earns its cost.

Does full autonomy make sense outside regulated industries?

The same logic applies anywhere an action is costly to reverse. Regulated industries simply make the cost visible sooner, through complaints, remediation and supervisors' questions.

Decide what your agents may do. Prove it. Enforce it.

Pryme Intelligence certifies each agent on your own cases, checks every action against five bounds at the moment it happens, and files every verdict.