Rate limiting, 429 and Retry-After

A rate limit caps how many requests one caller may make in a period. It protects capacity (one runaway script cannot starve everyone else), keeps costs predictable, and slows down credential stuffing and scraping. The limit is keyed on whatever identifies the caller best: an API key or user id for authenticated traffic, an IP address only as a fallback, since many users can share one address.

Saying no: 429

RFC 6585 defines 429 Too Many Requests for a caller that has sent too many requests in a given amount of time. The response should explain the condition and may carry Retry-After, which RFC 9110 defines as either a number of seconds or an HTTP date:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/problem+json

{"type": "https://api.example.com/problems/rate-limited",
 "title": "Too many requests", "status": 429,
 "detail": "Limit is 100 requests per minute for this key."}

Keep 429 distinct from 503 Service Unavailable: 429 says "you are over your limit", 503 says "the service is overloaded or down for everyone". Many APIs also report the budget on every response so clients can slow down before they hit it. X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset are a widespread convention, not a standard; the IETF HTTP APIs working group is drafting standard RateLimit-Policy and RateLimit fields, which at the time of writing are still an Internet-Draft, not an RFC.

Algorithms

• Fixed window — count requests per calendar minute and reset at the boundary. Cheap, one counter per key, but a caller can spend a full window's budget at the end of one minute and another at the start of the next: twice the limit within a second.
• Sliding window — count requests over the last N seconds from now, either by keeping timestamps (exact, more memory) or by weighting the previous window's count (approximate, cheap). No boundary burst.
• Token bucket — a bucket holds up to capacity tokens and refills at a steady rate; each request takes one, and an empty bucket means 429. It allows short bursts up to the capacity while holding the long-run rate, which is usually what an API wants.

1function fixedWindow(limit, windowSec) {2  const counts = new Map();3  return (t) => {4    const w = Math.floor(t / windowSec);5    const n = (counts.get(w) ?? 0) + 1;6    counts.set(w, n);7    return n <= limit;8  };9}10 11function tokenBucket(capacity, perSecond) {12  let tokens = capacity;13  let last = 0;14  return (t) => {15    tokens = Math.min(capacity, tokens + (t - last) * perSecond);16    last = t;17    if (tokens < 1) return false;18    tokens -= 1;19    return true;20  };21}22 23export function run() {24  // Five requests at second 59, five more at second 60.25  const times = [59, 59, 59, 59, 59, 60, 60, 60, 60, 60];26  const fw = fixedWindow(5, 60);27  const tb = tokenBucket(5, 5 / 60);28  return {29    fixedWindowAllowed: times.filter((t) => fw(t)).length,30    tokenBucketAllowed: times.filter((t) => tb(t)).length,31  };32}

With a token bucket, Retry-After falls out of the arithmetic: the time until the bucket holds one whole token, rounded up to whole seconds — rounding down tells the client to retry while it will still be refused.

Across many servers

Behind a load balancer, each instance counting on its own multiplies the limit by the number of instances. Real limiters keep the counter or bucket in a shared store (commonly Redis) and update it atomically, or enforce limits at the gateway in front of the instances.

The client's side

A good client treats 429 as an instruction, not an error: wait at least Retry-After, then retry; without it, back off exponentially with jitter. Retrying immediately in a loop only burns the next window too.

Common mistakes

• A fixed window with no burst protection, letting a caller send twice the limit across a boundary, as the example shows.
• Rounding Retry-After down, or sending 0, so the client retries into another 429.
• Per-instance counters behind a load balancer, which silently multiply the real limit.
• Returning 503 or 403 for a rate limit, so clients cannot tell "slow down" from "broken" or "forbidden".

What interviewers ask

• How would you rate-limit an API? — pick a key (API key or user), an algorithm (token bucket or sliding window), a shared atomic store, and respond 429 with Retry-After and limit headers.
• What is wrong with a fixed window? — bursts at the boundary allow up to twice the limit in a short span.
• 429 or 503? — 429 for one caller over its quota; 503 when the service itself cannot handle requests.