What is explainability?

Explainability is the ability to give an understandable reason for a specific output or decision an AI system produced, in terms the recipient can use. A customer needs to know why their request was declined; a reviewer, which facts drove a recommendation; an auditor, why an action was permitted. Each is a different explanation of the same event.

Why does explainability matter once agents act?

When an agent acts on someone's account, that person may need a reason. In some cases the law requires one. In the US, ECOA and Regulation B require specific reasons when credit is denied, including when a model is used. Under GDPR, people subject to decisions based solely on automated processing with legal or similarly significant effects have rights under Article 22. A business that cannot say why its agent acted is exposed.

The difficulty is that a model's internal reasoning is hard to inspect, and a model asked to explain itself may give a plausible account that is not what happened. For agents, the more reliable explanation comes from outside the model: which facts it used, where each came from, which rule applied and which bound decided. That record is part of agent governance is a bound, not a model.

How does explainability work in practice for agents?

A useful explanation of an agent's action draws on four things:

• The inputs: the request and the facts the agent used, each with its source.
• The rule: the policy or criteria the agent applied.
• The bounds: which limits were checked and which one decided the verdict.
• The person: for held actions, who approved and on what basis.

Different audiences get different cuts. The customer gets the main reasons in plain language. The reviewer gets the sources and the rule. The auditor gets the full verdict. Giving reasons under the EU AI Act is discussed in EU AI Act explainability: giving reasons for an agent's decision.

What does that look like in an example?

An online lender's agent prepares decisions on small business overdraft increases, and a credit officer approves them. An applicant is declined and asks why. The explanation is drawn from the record: the agent used the applicant's last six months of account data from the core banking ledger, applied the lender's policy on average balance against requested limit, and found the average balance below the policy threshold. The credit officer reviewed and agreed. The applicant receives the main reason in a sentence. Nobody had to ask the model what it was thinking.

How is explainability different from transparency and interpretability?

Transparency is openness in general: that AI is used, what it does, what data it relies on. Explainability is about a specific decision. A business can be fully transparent about using an agent for complaints and still be unable to explain one particular outcome.

Interpretability is narrower still: understanding how a model's internal mechanics produce outputs. It is a research field and a technical aid. For most business purposes, an explanation built from the agent's record is more useful and more dependable than one built from the model's internals.

Can an explanation reveal manipulation?

Yes, and this is an underrated benefit. When an agent does something strange, the record of its inputs often shows why: an instruction hidden in a document, or a message crafted to push it past its role. Recognising those patterns is covered in what is a jailbreak? An agent whose actions can be explained from its record is also an agent whose manipulation can be found.