Rate Limiting
Traffic ManagementCaps how many requests a client/key can make in a window, protecting systems from overload and abuse.
Rate limiting enforces a ceiling on request volume per key (per user, IP, API key, or endpoint) over time, protecting downstream systems from being overwhelmed (accidentally by bugs/retries, or intentionally by abuse/DDoS) and enabling fair usage/tiered pricing. The core algorithms are: Fixed Window Counter (simple, but bursty at window edges), Sliding Window Log/Counter (smoother, more accurate, more memory), Token Bucket (allows bursts up to bucket size, refills at a steady rate — most commonly used in practice), and Leaky Bucket (smooths bursts into a constant output rate, good for shaping traffic to a fixed downstream capacity).
How it connects
Rate Limiting as the source, with the components it typically interacts with.
- → API Gateway: Rate limiting is almost always implemented as gateway middleware so it applies uniformly across all routed services.
- → Load Balancer: Some rate limiting happens at the LB/edge layer (e.g. per-IP connection limits) before requests are even distributed to backends.
- → Caching: Token-bucket/sliding-window counters are usually stored in a fast shared cache (Redis) so limits are enforced consistently across many gateway/API instances.