# Load Balancing

A load balancer is the component that turns one service into many without the client knowing. It is also where TLS terminates, where client IPs are lost, and where "the site is slow" is often actually "one backend is sick".

## What it does

| Job | Detail |
|---|---|
| **Distribute** | Spread requests across healthy backends |
| **Detect failure** | Health checks remove bad backends |
| **Terminate TLS** | Offload cryptography, centralise certificates |
| **Preserve sessions** | Optionally pin a client to one backend |
| **Observe** | Access logs, latency, error rates per backend |

## Layer 4 vs layer 7

| | L4 (TCP/UDP) | L7 (HTTP) |
|---|---|---|
| Sees | IP, port | Method, path, headers, cookies |
| Can route by | Connection tuple | URL, host, header, weight, canary |
| Cost | Cheap, fast, opaque | More CPU, more capability |
| Client IP | Preserved or PROXY protocol | `X-Forwarded-For` / `Forwarded` |
| Typical use | Databases, TCP services, raw throughput | Web apps, APIs, gRPC |

Most web architectures use L7 at the edge and L4 for internal services.

## Algorithms

| Algorithm | Good for | Bad for |
|---|---|---|
| **Round robin** | Homogeneous backends, stateless requests | Heterogeneous capacity |
| **Least connections** | Variable request duration | Very short requests |
| **Least response time** | Latency-sensitive | Noisy neighbours |
| **Consistent hashing** | Caches, sharded state | Rebalancing cost on change |
| **Random with two choices** | Large fleets, simple | Predictable stickiness |
| **Weighted** | Canary, mixed instance sizes | Manual tuning |

## Health checks are the part that matters

A health check answers one question: *should this backend receive traffic?* A bad one causes outages rather than preventing them.

- **Shallow** (`GET /` returns 200) — cheap, catches a dead process, misses a dead database.
- **Deep** (`/healthz` checks dependencies) — catches more, and can take the whole fleet out at once when a shared dependency blips.
- **Thresholds** — `unhealthy_threshold` too low causes flapping under load; too high keeps sending traffic to a dead host.

Best practice: a shallow liveness check for routing, a separate readiness check that gates new traffic, and dashboards that distinguish "no backends healthy" from "all backends slow".

## Sessions and stickiness

If the application keeps state in memory, the balancer must pin the client (cookie or source-IP hash). Pinning:

- concentrates load unevenly as clients come and go,
- makes deploys disruptive (pinned clients hit a draining backend),
- hides the real fix, which is externalising session state.

## Where it goes wrong

- **Health check hits a cached page.** Backend is dead, cache returns 200, traffic keeps flowing to nothing.
- **Health check too deep.** One database restart marks every backend unhealthy and the balancer returns 503 to everyone.
- **Client IP lost.** Rate limits, geo rules and audit logs now see the balancer. Add `X-Forwarded-For` and trust it only from the balancer's own ranges.
- **TLS terminated twice, or not re-encrypted internally.** Both are defensible; doing one by accident is not.
- **Sticky sessions during a deploy.** Draining with a long timeout, or none, drops in-flight requests.
- **No connection limits per backend.** One slow client with many connections starves everyone (a real DoS vector).
- **Load balancer is a single point of failure.** It needs its own redundancy — DNS, anycast, or an active/standby pair with a floating VIP.

## Cloud equivalents

| Concept | AWS | Azure | GCP |
|---|---|---|---|
| L4 | Network Load Balancer | Load Balancer (Standard, L4) | Network LB |
| L7 | Application Load Balancer | Application Gateway | HTTP(S) LB |
| Global anycast | Global Accelerator | Front Door | Global external LB |

The names differ; the questions do not: which layer, which health check, how is the client IP preserved, and what happens when it fails.

## Teaching hook

**The supermarket checkout analogy.** One queue feeding several tills (least connections) versus a fixed queue per till (round robin). A till with a trainee cashier (slow backend) needs fewer customers, not the same number. And if the supervisor keeps sending people to a closed till because the "open" sign is still lit, that is a shallow health check.

Demo: run three backends with `python3 -m http.server`, put `nginx` or `caddy` in front, kill one, and watch traffic stop arriving. Then change the health check path and watch the behaviour change.

## Related Notes

- [[Core-Protocols/TLS-And-HTTPS.md]] — where certificates live
- [[Core-Protocols/HTTP-Versions-And-QUIC.md]] — protocol negotiation at the edge
- [[Network-Services/NAT-And-PAT.md]] — the older, cruder version of this
- [[Cloud-Modern-Networking/VPC-Fundamentals.md]] — cloud load balancer placement