How to rate-limit an AI agent at the API gateway

Set limits per agent, per route and per capability, and measure three things, not one: calls per window, value moved per window, and distinct customers or records touched. Throttle reads when a limit is hit, hold writes rather than drop them, and trip a breaker that drops the agent to read-only if it keeps pushing. Log every throttle and breach.

Why rate-limit an agent differently from an app, and who owns the limits?

Human-facing apps are limited to protect the system from load. Agents need limits for a second reason: to cap the damage a wrong decision can do before anyone notices. An agent that misreads a policy will apply the misreading at machine speed, across every case in its queue. A limit turns an incident affecting thousands into one affecting a handful.

Agents also behave differently under pressure. They retry on timeouts, split one task into many calls, and sometimes loop. A limit tuned for a mobile app will be either too loose to catch a runaway agent or too tight for normal work.

The integration team owns the mechanism on the gateway; the setup is in how to put an API gateway in front of an AI agent's tools. Process owners set the numbers for their agents' capabilities. Risk sets the ceiling no process owner can exceed.

How do you set the limits, step by step?

1. Process owner: measure normal volume. Replay a typical week of cases with the agent at read and count calls per route, value per route, and distinct customers touched. Evidence: a baseline per capability.
2. Process owner: set three limits per capability. Count per window; value per window for routes that move money; reach per window, meaning distinct customers or records changed. Start near the baseline, not at a round multiple of it.
3. Risk owner: set ceilings. No capability's limits may exceed them without a recorded exception.
4. Integration team: key limits to the agent's identity and the route. A shared key across agents lets one noisy agent starve the others and hides which one misbehaved.
5. Integration team: define what happens at the limit. Reads are throttled with a retry-after signal. Writes are held, not dropped, so no customer request is silently lost. Evidence: throttle and held entries.
6. Integration team: add a breaker. If an agent hits limits repeatedly in a short period, or its refusal rate spikes, the breaker drops it to depth 1 and alerts the process owner. Evidence: breaker events with the trigger.
7. Person, process owner: reviews and resets the breaker. Only a person resets it, after reading why it tripped.
8. Process owner: review limits at each certification. Raise them with evidence; lower them when volume falls.

Which controls matter, and what evidence do they leave?

Value limits on money routes. A count limit of 100 refunds an hour says nothing if each could be large. Evidence: running value total on each verdict.

Reach limits. Many agent failures are wide, not deep: a small wrong change applied to many customers. Counting distinct customers touched catches that pattern. Evidence: reach totals per window.

Hold, do not drop. A dropped write is a customer request that vanished. Evidence: every over-limit write appears as held with a reference.

Per-agent identity. Evidence: limits and breaches attributed to one agent.

Breaker with human reset. Evidence: trip time, trigger, reset time and the person who reset it.

Retries count. An agent retrying a failed call is still calling. Count retries against the limit, and pair them with idempotency keys so a retry never doubles a payment; this matters most when you connect an agent to your payment gateway.

What should you measure?

• Throttles and holds per capability. Steady throttling means the limit is below real demand or the agent is inefficient.
• Breaker trips, and what caused each.
• Held writes awaiting approval. If the queue grows faster than people clear it, the limit is sending work to people who cannot absorb it. Watch it alongside the agent's escalation rate.
• Calls per completed case. A rising number often means the agent is looping.
• Limit changes over time, with who approved each.

Where does it usually go wrong?

Only counts are limited. The agent stays under its call limit while moving far more value than anyone intended.

Limits are set generously "to avoid friction". A limit that never fires protects nothing. Set it close to normal and raise it on evidence.

Writes are rejected with an error. The agent reports failure, the customer is told to try again, and the request is lost. Hold instead.

The gateway is mistaken for the whole control. A gateway counts and throttles; it does not decide whether an action was justified. The difference is explained in whether an agentic operating system is the same as an AI gateway.

Agents that do not use APIs escape the limit. An agent driving a screen or a browser bypasses the gateway entirely. If your agent works that way, see whether an AI agent can work without an API and put equivalent limits on that path.

Where Pryme fits

In Pryme, the customer's own limits are one of the five bounds checked on every action, alongside role scope, certificate, sources and platform guardrails. Over-limit writes become held verdicts, and every throttle and hold is filed. The governed agentic operating system guide explains how limits relate to depth and certification.