API Rate Limiting with Token Buckets and Redis: What Actually Works

Why I finally stopped shipping APIs without limits

The first paid API I put on the public internet had auth, validation, and careful indexes. It did not have rate limits. A partner script with a retry loop stuck on a 500 burned through our MongoDB connection pool in under four minutes. CPU looked fine. Latency exploded. Customers saw timeouts.

That incident taught me a blunt rule: every externally reachable endpoint needs a budget. Not just auth. Auth tells you who is calling. Rate limits tell you how hard they can hit you. Without both, one bad client or leaked key can flatten everyone else.

I now treat rate limiting as part of the default middleware stack, same as CORS and request logging. The algorithm matters less than having something consistent, observable, and hard to bypass by opening a second process.

If you run multi-tenant SaaS, rate limits are part of fairness between tenants. Without them, one noisy customer becomes everyone else's latency problem—and your status page.

Token bucket vs fixed window vs sliding window

Fixed windows are simple: allow N requests per minute, reset on the clock. They fail at the boundary. A client can send N requests at 00:59 and N more at 01:00 and get 2N in two seconds. I have watched scrapers do exactly that against marketing APIs.

Sliding windows fix the burst at the cost of more bookkeeping. Token buckets sit in a sweet spot for APIs: you refill tokens at a steady rate, and you can allow short bursts up to a capacity. That matches how real UIs behave—idle, then a flurry of clicks.

I default to token bucket for product APIs. I use a strict fixed or sliding window only when a contract literally says “100 requests per calendar minute” and auditors care about the wall clock.

• Fixed window: easy, cheap, bursty at boundaries
• Sliding window: smoother, more Redis ops or memory
• Token bucket: burst-friendly, good for interactive clients

Redis-backed token bucket in Node

In-memory maps work on a single Node process and fall apart behind a load balancer. Redis is the shared counter I trust. The pattern below uses a small Lua script so refill and consume are atomic. Without atomicity you under-count under concurrency and clients sneak past the limit.

I key by authenticated user id when present, otherwise by IP plus route group. Never key only by IP for logged-in APIs—NAT and mobile carriers share addresses and you will punish the wrong people.

const REDIS_SCRIPT = `
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_per_ms = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local data = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(data[1])
local ts = tonumber(data[2])

if tokens == nil then
  tokens = capacity
  ts = now
end

local elapsed = math.max(0, now - ts)
tokens = math.min(capacity, tokens + elapsed * refill_per_ms)

if tokens < cost then
  redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
  redis.call('PEXPIRE', key, math.ceil(capacity / refill_per_ms))
  return {0, tokens}
end

tokens = tokens - cost
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', key, math.ceil(capacity / refill_per_ms))
return {1, tokens}
`;

async function takeToken(redis, key, { capacity, refillPerSec, cost = 1 }) {
  const now = Date.now();
  const refillPerMs = refillPerSec / 1000;
  const [allowed, remaining] = await redis.eval(
    REDIS_SCRIPT, 1, key, capacity, refillPerMs, now, cost
  );
  return { allowed: allowed === 1, remaining: Number(remaining) };
}

Wiring it as Express middleware

I keep limits per route class, not one global number. Auth login stays tight. Read-heavy list endpoints get more headroom. Expensive report generation costs multiple tokens so one call counts like five cheap GETs.

Always return 429 with Retry-After when you can estimate it. Clients that respect the header stop hammering. Also emit RateLimit-Remaining style headers so partner integrations can back off before they fail.

I unit-test middleware with a fake Redis that records eval calls. That catches key shape bugs before load tests.

function rateLimit({ name, capacity, refillPerSec, cost = 1 }) {
  return async (req, res, next) => {
    const subject = req.user?.id || req.ip;
    const key = `rl:${name}:${subject}`;
    try {
      const result = await takeToken(redis, key, { capacity, refillPerSec, cost });
      res.setHeader('X-RateLimit-Limit', String(capacity));
      res.setHeader('X-RateLimit-Remaining', String(Math.floor(result.remaining)));
      if (!result.allowed) {
        res.setHeader('Retry-After', '1');
        return res.status(429).json({ error: 'rate_limit_exceeded' });
      }
      return next();
    } catch (err) {
      // Fail open or closed? See next section.
      console.error('rate limit redis error', err);
      return next();
    }
  };
}

app.post('/auth/login', rateLimit({ name: 'login', capacity: 10, refillPerSec: 0.2 }), loginHandler);
app.get('/api/items', rateLimit({ name: 'items', capacity: 120, refillPerSec: 2 }), listItems);

Fail open, fail closed, and the Redis outage

When Redis blips, you choose: fail open (serve traffic, lose protection) or fail closed (reject everyone). Early on I failed open and a Redis restart coincided with a crawler spike. We stayed up for healthy users but the crawler enjoyed unrestricted access for three minutes.

For login, password reset, and anything billing-related I now fail closed with a clear 503. For low-risk public GETs I fail open and page ops. Document the choice. Do not leave it as an accidental catch that always calls next().

Also put a local circuit breaker around Redis so a dead cluster does not add 200ms of timeout latency to every request while Node waits on the client.

Testing and the mistakes I still see

Unit-test the Lua script with concurrent callers. I use a small Node script that fires 200 parallel requests against a local Redis and asserts the total allowed count stays within capacity plus a tiny float tolerance.

Load-test with realistic keys. A limit of 100/min “works” in staging when everyone shares one test user and fails in production when each user has their own bucket and aggregate load is higher.

• Do not rate-limit health checks on the same key as user traffic
• Separate anonymous and authenticated buckets
• Log 429s with route and subject hash, not raw tokens
• Watch p95 Redis latency—slow rate limits become your new bottleneck
• Document partner-facing limits in the API docs before they guess

What I ship by default now

Token bucket in Redis, atomic Lua, per-route profiles, Retry-After on 429, and an explicit fail-open/fail-closed policy. That stack has survived partner bursts, leaked keys, and one very enthusiastic QA script.

Start stricter than you think, then raise limits with data. Raising a limit is a config change. Cleaning up after an unbounded endpoint is an incident.

Choosing numbers that survive contact with users

Limits look scientific in a design doc and arbitrary in production. I start from a capacity estimate: peak legitimate RPS for a single user or API key, times a safety factor of three, then round to something explainable in docs. Login might be 5 attempts per minute. A mobile app sync endpoint might be 60 per minute with a burst of 20.

I also cost expensive routes higher. A PDF export that hits the database and an object store might cost 10 tokens while a health-style metadata GET costs 1. That keeps the same bucket model without inventing a second limiter.

Document the limits publicly for partner APIs. Undocumented limits look like random outages. When we raised a partner from 60 to 300 after reviewing their traffic shape, the change was a config deploy—not an emergency rewrite.

• Derive limits from observed p95 client behavior plus headroom
• Charge multi-token costs for expensive handlers
• Publish partner limits and changelog them
• Alert on 429 rate, not only on 5xx

Edge rate limits vs application rate limits

CDN or API-gateway limits catch volumetric abuse early and protect origin CPU. Application limits enforce product rules per user and per route. I run both when traffic warrants it. Edge alone cannot express “this authenticated tenant gets 10 report jobs per hour.”

If you only put limits in Nginx and forget the app, a second ingress path—internal service mesh, forgotten port, new cloud load balancer—bypasses them. Defense in depth means the app still knows how to say no.

Key takeaways

• Put rate limits on every public endpoint before launch, not after an incident.
• Prefer Redis-backed token buckets for multi-instance APIs; keep consume/refill atomic.
• Key by user id when authenticated; use IP only as a fallback for anonymous traffic.
• Return 429 with Retry-After and remaining-limit headers clients can honor.
• Decide fail-open vs fail-closed per route class when Redis is unavailable.
• Load-test with many subjects—shared staging keys hide real aggregate load.