- Prevents abuse and denial-of-service attacks
- Protects backend resources and ensures fair usage
- Helps avoid unexpected costs and keeps APIs reliable
- When a client exceeds the allowed rate, return HTTP status 429 Too Many Requests
- Include a
Retry-Afterheader to tell the client when to try again
Example response:
HTTP/1.1 429 Too Many Requests
Retry-After: 10
Content-Type: application/json
{"error": "Rate limit exceeded. Try again later."}There are many rate limiting algorithms to choose from, but two of the more common ones are Sliding Window Log (Rolling Window) and Token Bucket.
Sliding Window Log (rolling window):
- For each API key, we store timestamps of recent requests (up to the limit).
- When a request arrives, we remove timestamps older than now - window_seconds.
- If fewer than limit timestamps remain, we allow the request and append the current timestamp.
- Otherwise, we deny the request with HTTP 429 and tell the client how long to wait using Retry-After.
# Suppose rate limit is 5 per 5 minutes
# Times of last 5 requests
[10:11, 10:11, 10:12, 10:13, 10:14]
# If a new request comes in at 10:15
# Remove all requests older than 10:10
# Check if less than limit, if not return 429
# Next request comes in at 10:17
# Remove both requests that are before 10:12
# Since the list length is less than rate limit, the request can be handled, and the time added to the end of list (deque)
Token Bucket:
- Each API key starts with a bucket of tokens (capacity). Each request costs 1 token.
- On every request, we refill tokens based on how much time has passed since the last refill, up to the maximum capacity.
- If at least 1 token is available, we consume one and allow the request.
- If no tokens are available, we deny the request with HTTP 429 and include Retry-After.
# A key with 10 max tokens that refills each minute
# two requests come in at 10:10, we now have 8 tokens left
# 8 more requests come in at 10:11, since a minute has gone by,
# we first add 1 token (now 9), then subtract 8 to have 1 left.
These examples store rate limit state in memory. In production across multiple servers, rate limiting is usually enforced by an API gateway or a shared store like Redis.
When an API returns HTTP 429 Too Many Requests, it means the client is sending requests faster than allowed. A client-side retry is logic in the client application that waits for a short period of time and then tries the request again instead of immediately failing.
Many APIs include a Retry-After header in the response, which tells the client how many seconds to wait before retrying. This helps clients behave politely, reduces unnecessary load on the server, and improves reliability for temporary rate-limit errors.
Only retry idempotent requests like GET. Be careful retrying POST/PUT that change state, as running those types of requests more than once (like deducting an account balance) can be a bug. Retries on getting an account balance are safer.
import time
from collections import defaultdict, deque
RATE_LIMIT = 10 # requests
WINDOW = 10 # seconds
request_log = defaultdict(deque)
def sliding_window_check(api_key: str):
# time.monotonic() is more accurate than time.time()
now = time.monotonic()
log = request_log[api_key]
# Remove timestamps outside the window
cutoff = now - WINDOW
while log and log[0] < cutoff:
log.popleft()
if len(log) < RATE_LIMIT:
log.append(now)
return True, 0
# Oldest request still in-window determines when we can retry
retry_after = int((log[0] + WINDOW) - now) + 1
return False, max(retry_after, 1)import time
from dataclasses import dataclass
@dataclass
class TokenBucket:
capacity: float
refill_rate_per_sec: float # tokens per second
tokens: float
last_refill: float
def refill(self, now: float):
elapsed = now - self.last_refill
if elapsed > 0:
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate_per_sec)
self.last_refill = now
def consume(self, amount: float = 1.0):
now = time.monotonic()
self.refill(now)
if self.tokens >= amount:
self.tokens -= amount
return True, 0
# time until we have `amount` tokens
needed = amount - self.tokens
retry_after = int(needed / self.refill_rate_per_sec) + 1
return False, max(retry_after, 1)
from collections import defaultdict
# Example: capacity 10, refill 1 token per second
buckets = defaultdict(lambda: TokenBucket(
capacity=10,
refill_rate_per_sec=1.0,
tokens=10.0,
last_refill=time.monotonic()
))
from flask import Flask, request, jsonify
app = Flask(__name__)
def get_api_key():
key = request.headers.get("X-API-Key")
if not key:
return None
return key
@app.route("/data")
def data():
api_key = get_api_key()
# replace code below with real API key validation
if not api_key:
return jsonify({"error": "missing_api_key"}), 401
# Choose one strategy for the lab demo:
allowed, retry = sliding_window_check(api_key)
# allowed, retry = buckets[api_key].consume()
if not allowed:
resp = jsonify({"error": "rate_limited", "retry_after": retry})
resp.status_code = 429
resp.headers["Retry-After"] = str(retry)
return resp
return jsonify({"data": "ok"})for i in {1..12}; do
code=$(curl -s -o /dev/null -w "%{http_code}" -H "X-API-Key: key1" http://localhost:5000/data)
echo "$i -> $code"
done
- This prints only the HTTP status codes for each request. Add a sleep command as needed to test burst vs. spaced requests.
For this challenge, update your existing networking API (the one with API-key authentication) to include rate limiting on all endpoints that accept API keys. You can use either the sliding window log or token bucket method from above.
- Return HTTP 429 and a Retry-After header when the limit is exceeded.
- Test your implementation using curl as shown above.
- Make sure your rate limiting logic is per API key, not global.
- Bonus: Implement the JS retry logic for idempotent requests if you have a frontend.
This matches how real-world APIs protect themselves from abuse and ensures fair usage for all clients!
- Fixed window can be bursty at edges
- Sliding window log is fairer but stores timestamps (uses more memory)
- Token bucket allows controlled bursts and smooth refill
- All examples here are in-memory; for production, use Redis or a distributed store
For idempotent requests, you can retry after the server's suggested delay:
async function fetchWithRetry(url, options, attempts = 3) {
const res = await fetch(url, options);
if (res.status === 429 && attempts > 0) {
const retry = Number(res.headers.get('Retry-After') || 1);
await new Promise(r => setTimeout(r, retry * 1000));
return fetchWithRetry(url, options, attempts - 1);
}
return res;
}- defaultdict: a dictionary that auto-creates a default value when a new key is accessed
- deque: a fast queue-like list where we efficiently remove from the front and add to the end
- buckets: a dictionary mapping each API key to its own TokenBucket object (rate limit state)
Important concept:
- Rate limiting does not validate API keys. We validate the API key first.
- After validation, we use the API key string as the identifier to track per-key rate limiting state.
- The first time we see a valid key, defaultdict automatically creates an empty deque (sliding window) or a TokenBucket instance (token bucket).
This lab teaches practical API rate limiting and how to handle 429 responses in real clients!