Starphix
Network Engineering / Sources
The Lab / Sources / Network Engineering / Network-Services/Load-Balancing.md

Load Balancing

A load balancer is the component that turns one service into many without the client knowing. It is also where TLS terminates, where client IPs are lost, and where "the site is slow" is often actually "one backend is sick".

What it does

Job Detail
Distribute Spread requests across healthy backends
Detect failure Health checks remove bad backends
Terminate TLS Offload cryptography, centralise certificates
Preserve sessions Optionally pin a client to one backend
Observe Access logs, latency, error rates per backend

Layer 4 vs layer 7

L4 (TCP/UDP) L7 (HTTP)
Sees IP, port Method, path, headers, cookies
Can route by Connection tuple URL, host, header, weight, canary
Cost Cheap, fast, opaque More CPU, more capability
Client IP Preserved or PROXY protocol X-Forwarded-For / Forwarded
Typical use Databases, TCP services, raw throughput Web apps, APIs, gRPC

Most web architectures use L7 at the edge and L4 for internal services.

Algorithms

Algorithm Good for Bad for
Round robin Homogeneous backends, stateless requests Heterogeneous capacity
Least connections Variable request duration Very short requests
Least response time Latency-sensitive Noisy neighbours
Consistent hashing Caches, sharded state Rebalancing cost on change
Random with two choices Large fleets, simple Predictable stickiness
Weighted Canary, mixed instance sizes Manual tuning

Health checks are the part that matters

A health check answers one question: should this backend receive traffic? A bad one causes outages rather than preventing them.

  • Shallow (GET / returns 200) — cheap, catches a dead process, misses a dead database.
  • Deep (/healthz checks dependencies) — catches more, and can take the whole fleet out at once when a shared dependency blips.
  • Thresholds — unhealthy_threshold too low causes flapping under load; too high keeps sending traffic to a dead host.

Best practice: a shallow liveness check for routing, a separate readiness check that gates new traffic, and dashboards that distinguish "no backends healthy" from "all backends slow".

Sessions and stickiness

If the application keeps state in memory, the balancer must pin the client (cookie or source-IP hash). Pinning:

  • concentrates load unevenly as clients come and go,
  • makes deploys disruptive (pinned clients hit a draining backend),
  • hides the real fix, which is externalising session state.

Where it goes wrong

  • Health check hits a cached page. Backend is dead, cache returns 200, traffic keeps flowing to nothing.
  • Health check too deep. One database restart marks every backend unhealthy and the balancer returns 503 to everyone.
  • Client IP lost. Rate limits, geo rules and audit logs now see the balancer. Add X-Forwarded-For and trust it only from the balancer's own ranges.
  • TLS terminated twice, or not re-encrypted internally. Both are defensible; doing one by accident is not.
  • Sticky sessions during a deploy. Draining with a long timeout, or none, drops in-flight requests.
  • No connection limits per backend. One slow client with many connections starves everyone (a real DoS vector).
  • Load balancer is a single point of failure. It needs its own redundancy — DNS, anycast, or an active/standby pair with a floating VIP.

Cloud equivalents

Concept AWS Azure GCP
L4 Network Load Balancer Load Balancer (Standard, L4) Network LB
L7 Application Load Balancer Application Gateway HTTP(S) LB
Global anycast Global Accelerator Front Door Global external LB

The names differ; the questions do not: which layer, which health check, how is the client IP preserved, and what happens when it fails.

From the Network Engineering research vault · CC BY 4.0 · all notes · raw markdown