Rate limits and quotas

Limits are enforced at the gateway, before a request reaches any service.

Two kinds of ceiling

Kind	Enforced by	Exceeding it
Rate limit	Requests per interval, per credential	`429`
Plan limit	Resource or action counts on your plan	`402 upgrade_required`
Credit balance	Spendable balance	`402 insufficient_credits`

A 429 is transient — wait and retry. A 402 is not; retrying changes nothing.

Per-credential limits

Monthly request quotas scale with plan tier and are enforced per API key.

Quotas are per key, not aggregated per tenant. Minting several keys currently raises effective throughput — a per-tenant aggregate limit exists but is not yet applied. Do not design around this: it will change.

Per-memory limits

Requests to a memory's own hostname carry their own ceilings — roughly 600 per minute, 30,000 per hour and 1,000,000 per month, per memory.

These sit alongside the per-key limits rather than replacing them.

Backing off

Honour Retry-After when it is present, and add jitter when it is not. Retrying a whole fleet on a fixed interval turns one limit breach into a synchronised thundering herd.

const after = Number(res.headers.get('retry-after'));
const delay = Number.isFinite(after) && after > 0
  ? after * 1000
  : 2 ** attempt * 250 + Math.random() * 250;

Staying under

Batch ingestion. One request with a whole document beats twenty with fragments — and retrieves better.

Cache reads. Workspace and memory metadata changes rarely; a short client-side cache removes most of that traffic.

Let search cache work. Repeat queries are served from cache and invalidated automatically when new content lands, so re-asking is cheap and still correct.

Stream long answers. Streaming does not reduce request count, but it removes the temptation to poll.

Alert on budget. Notifications fire at 80% and 100% of the period grant — wire them somewhere a human reads.