Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Rate limit (API)

A rate limit caps how much an account can use an API in a time window, typically as requests per minute (RPM) and tokens per minute (TPM). Requests over the limit are rejected, usually with an HTTP 429 error.

Also called: rate limiting, RPM, TPM

By Divya Sharma · Checked Nov 16, 2026 · How we review

AI providers limit usage to share capacity fairly and to prevent abuse. Limits are usually set per organisation or project and per model, and they rise as your account builds usage history. Anthropic's documentation, for example, measures Messages API limits in requests per minute, input tokens per minute and output tokens per minute, and returns a 429 error saying which limit was exceeded.

How limits behave

A per-minute limit is often enforced over shorter intervals, so a burst can fail even if the minute's total would be fine. Anthropic says its API uses a token bucket, so capacity is continuously replenished rather than reset at fixed times. Some limits count only uncached input, which makes prompt caching a way to fit more work under the same limit. Separate spend caps can also block requests once a monthly budget is reached.

Handling 429 errors

Read the `retry-after` header when present and wait that long. Otherwise retry with exponential backoff and some random jitter, and cap the number of retries so a failing job does not loop forever. Response headers usually report how much of each limit is left, which lets a client slow down before hitting it. For large offline jobs, a batch API often has its own, higher limits.

Planning around limits

Spread traffic across time, keep prompts lean, and set your own per-workspace limits so one service cannot starve others. An LLM gateway can queue requests and route to a fallback model or a second provider when one is limited. Rate limits are separate from token pricing.

Related terms

Token pricing
Token pricing is how most AI model APIs charge: a set price per million input tokens you send and a separate, usually higher, price per million output tokens the model generates, with discounts for cached input and batch jobs.
Batch API
A batch API lets you submit a large set of model requests in one job and collect the results later, instead of waiting on each call. Providers run the job when capacity allows and usually charge less for it.
LLM gateway
An LLM gateway is a proxy server between your applications and model providers that exposes one API and handles keys, routing, fallbacks, rate limits, logging and spend tracking in a single place.
Model fallback
Model fallback is a rule that sends a request to a backup model or provider when the first choice fails, times out, hits a rate limit or rejects the input. It keeps an AI feature working through outages.
API key
An API key is a secret string an application sends with each request so that a service can tell which account is calling, apply its limits and bill it. Anyone holding the key can usually use it.

Sources