Load Balancer
Traffic ManagementDistributes incoming requests across a fleet of servers to maximize throughput and avoid overload.
A load balancer (LB) sits in front of a pool of servers and routes each incoming request to one of them, based on a chosen algorithm. It exists to solve one core problem: a single server has a ceiling on how many requests it can serve, and it is a single point of failure. By fanning traffic out across many identical servers, an LB turns a fleet into something that behaves like one very large, very durable server. LBs can operate at Layer 4 (TCP/UDP, routing based on IP/port, very fast, protocol-agnostic) or Layer 7 (HTTP, can route based on path/headers/cookies, terminate TLS, do content-based routing).
How it connects
Load Balancer as the source, with the components it typically interacts with.
- → API Gateway: API gateways usually sit behind (or fold in) a load balancer so a single fleet of gateway instances can be scaled horizontally while still presenting one virtual IP.
- → Caching: LBs often route based on consistent hashing so requests for the same key land on the same backend, maximizing local cache hit rate on that node.
- → Rate Limiting: Per-client rate limits are frequently enforced at the LB/edge tier so abusive traffic is rejected before it consumes backend capacity.
- → CDN (Content Delivery Network): A global/anycast CDN typically fronts regional load balancers, absorbing static traffic and routing dynamic requests to the nearest LB.
- → Consistent Hashing: Consistent hashing is the algorithm an LB uses to pick a backend so that adding/removing servers only reshuffles a small fraction of traffic.