Load Balancing
Distributes incoming traffic across multiple servers so no single machine becomes a bottleneck and failures are isolated.
Interview tip Lead with a 30-second definition, then one real system example and name 2–3 designs where Load Balancing is non-negotiable.
① What it is (30 seconds)
Distributes incoming traffic across multiple servers so no single machine becomes a bottleneck and failures are isolated.
Analogy: A restaurant host seating guests across waiters instead of one overloaded server.
② How it works in system design
Client hits a load balancer (L4 TCP or L7 HTTP). LB picks a healthy backend via round-robin, least connections, or weighted rules. Health checks remove dead nodes. SSL termination and sticky sessions optional.
Typical placement
Client→Edge / Gateway→Load→Services→Data stores
③ Concrete system design example
Scenario: URL shortener at 50K RPS: 20 stateless API pods behind an AWS ALB. ALB terminates TLS, routes by path, drains unhealthy pods during deploy. Session-free redirects need no stickiness.
④ Important interview Q&A
| Question | Answer |
|---|---|
| L4 vs L7 load balancer? | L4 routes by IP/port (fast, WebSocket-friendly). L7 routes by URL, headers, cookies (microservices, canary). |
| What if one server is slow? | Least-connections beats round-robin when request durations vary widely. |
| How handle deploy without downtime? | Rolling deploy + health check grace period; LB stops sending to old version. |
⑤ Seen in these system designs
- URL Shortener — redirect hot path
- API Gateway — edge routing
- WhatsApp & Chat — WebSocket LB
In interviews, after explaining the concept, say: "This shows up directly in …" and link two designs.
⑥ Revision checklist
- Named L4 vs L7
- Health checks
- SSL termination
- Stateless vs sticky
- Global + regional LB