System Design Fundamentals
Core building blocks every senior interview expects — scalability, CAP, caching, load balancing, databases, CDN, and estimation.
Interview tip Start every answer: requirements (functional + non-functional) → estimate scale → high-level diagram → deep dive 2 components → trade-offs.
① The 45-minute interview framework
Every system design answer follows the same skeleton:
- Clarify (5 min): functional requirements, non-functional (latency, availability, consistency, scale), out of scope
- Estimate (5 min): DAU, QPS, storage, bandwidth — round numbers, state assumptions
- High-level design (10 min): boxes and arrows — client, CDN, LB, services, cache, DB, async queues
- Deep dive (15 min): pick 2 components the interviewer cares about — data model, hot path, sharding
- Trade-offs (5 min): SQL vs NoSQL, sync vs async, strong vs eventual consistency
- Wrap-up (5 min): failure modes, monitoring, future scale
Say aloud: "I'll start with requirements and scale, then draw the architecture, then go deep on X and Y." Interviewers reward structure over jumping to Redis.
② Scalability — vertical vs horizontal
| Approach | How | Pros | Cons |
|---|---|---|---|
| Vertical scale | Bigger CPU/RAM/disk on one machine | Simple, no code changes | Hard ceiling, single point of failure, expensive at top tier |
| Horizontal scale | More machines behind a load balancer | Near-unlimited, fault tolerant | Needs stateless apps, data partitioning, ops complexity |
| Auto-scaling | Add/remove instances on metrics | Handles traffic spikes | Cold start latency, config tuning |
| Read replicas | Copy DB for read-heavy workloads | Offload reads from primary | Replication lag, write still on primary |
Stateless services scale easily: session data lives in Redis or DB, not in app memory. Sticky sessions reduce flexibility — prefer shared session store.
Stateful services (WebSocket rooms, in-memory counters) need sharding, consistent hashing, or dedicated state stores.
Stateful services (WebSocket rooms, in-memory counters) need sharding, consistent hashing, or dedicated state stores.
Scaling decision tree
Traffic growing?→Optimize first→Cache + CDN→Scale reads→Scale writes
③ Back-of-envelope estimation
Assumptions
- 1M DAU × 10 requests/day = 10M req/day ≈ 116 req/s average
- Plan for 10× peak → ~1,200 req/s at peak
- Avg request 1 KB in + 5 KB out → 1.2K × 6 KB ≈ 7 MB/s bandwidth
- 100 bytes per record × 1B records = 100 GB raw (plus indexes ~2–3×)
- 1 photo 200 KB × 10M uploads/month ≈ 2 TB/month storage growth
| Resource | Formula | Example |
|---|---|---|
| QPS | DAU × actions/day ÷ 86,400 | 10M DAU × 20 ÷ 86,400 ≈ 2,300 avg |
| Storage | records × size × retention | 1B users × 500 B × 5 yr |
| Bandwidth | QPS × avg payload | 5K RPS × 50 KB = 250 MB/s |
| Memory (cache) | hot data % × total dataset | 20% of 100 GB = 20 GB Redis |
Rules of thumb: round to nearest power of 10; state assumptions ("assuming 10:1 read:write"); latency budget: CDN 10ms + LB 5ms + app 50ms + cache 1ms + DB 10ms = ~76ms p99 target.
④ Load balancing
| Algorithm | Behavior | Best for |
|---|---|---|
| Round robin | Rotate requests evenly | Homogeneous stateless servers |
| Least connections | Send to server with fewest active conns | Long-lived connections, varying request times |
| Weighted round robin | More traffic to stronger machines | Mixed hardware capacity |
| IP hash / consistent hash | Same client → same server | Session affinity without shared store (limited) |
| Layer 4 (TCP) | Route by IP/port, fast | Raw throughput, WebSocket |
| Layer 7 (HTTP) | Route by URL, headers, cookies | Microservices, A/B tests, canary |
Load balancer placement
Clients
DNS (geo-routing)
Global LB
Regional LB (L7)
App Server 1
App Server 2
App Server N
Health checks: LB polls
/health — remove unhealthy nodes within seconds. SSL termination at LB offloads crypto from app servers. Active-passive for stateful components; active-active for stateless.⑤ Caching strategies
| Pattern | Read path | Write path | Risk |
|---|---|---|---|
| Cache-aside | App → cache → miss → DB → populate cache | App writes DB, invalidates cache | Stale data if invalidation missed |
| Read-through | Cache fetches from DB on miss | Same as cache-aside writes | Cache library must support it |
| Write-through | Read from cache | Write cache + DB together | Write latency; cache and DB must sync |
| Write-behind | Read from cache | Write cache; async flush to DB | Data loss if cache crashes before flush |
Always set TTL — even with invalidation, TTL is your safety net. Eviction: LRU (common), LFU (frequency), TTL-based.
Hot key problem: one viral key hammers a single Redis shard — replicate hot keys, local in-process cache, or pre-warm.
Cache stampede: many requests miss simultaneously — use singleflight/locking so only one thread repopulates.
Hot key problem: one viral key hammers a single Redis shard — replicate hot keys, local in-process cache, or pre-warm.
Cache stampede: many requests miss simultaneously — use singleflight/locking so only one thread repopulates.
Cache-aside flow
Read request→Check Redis→Hit → return→Miss → DB→Set cache + return
⑥ CAP theorem & consistency
CAP: During a network partition, you choose between Consistency (all nodes see same data) and Availability (every request gets a response). Partition tolerance is non-negotiable in distributed systems — so you pick CP or AP.
| Choice | Example systems | Use when |
|---|---|---|
| CP (Consistency + Partition) | ZooKeeper, etcd, HBase | Banking, inventory, leader election |
| AP (Availability + Partition) | Cassandra, DynamoDB, CouchDB | Social feeds, shopping carts, analytics |
| Strong consistency | Single-leader replication, sync writes | Financial transactions, unique constraints |
| Eventual consistency | Async replication, quorum reads | User profiles, like counts, metrics |
PACELC extension: Else (no partition), choose Latency vs Consistency. Most web apps pick low latency with eventual consistency for non-critical reads.
Read-your-writes: user sees their own updates immediately — route to primary or use session stickiness + version checks.
Read-your-writes: user sees their own updates immediately — route to primary or use session stickiness + version checks.
⑦ Database types — when to use what
| Type | Examples | Strengths | Weak for |
|---|---|---|---|
| Relational (SQL) | PostgreSQL, MySQL | ACID, joins, complex queries | Massive horizontal write scale |
| Wide-column | Cassandra, HBase | High write throughput, time-series | Ad-hoc joins, transactions |
| Document | MongoDB, DynamoDB | Flexible schema, nested JSON | Multi-document ACID (improving) |
| Key-value | Redis, Memcached | Sub-ms reads, sessions, counters | Complex queries, durability (Redis) |
| Search | Elasticsearch, OpenSearch | Full-text, aggregations, logs | Primary transactional store |
| Graph | Neo4j, Neptune | Relationship traversals | Simple CRUD at huge scale |
| Vector | Pinecone, pgvector | Similarity search, RAG | Exact lookups by primary key |
Polyglot persistence: PostgreSQL for orders, Redis for sessions, Elasticsearch for search, S3 for blobs — each store optimized for its access pattern.
Sharding: split rows by user_id hash or range when single-node limits hit (~10K writes/sec PG, higher with Citus/Cockroach).
Sharding: split rows by user_id hash or range when single-node limits hit (~10K writes/sec PG, higher with Citus/Cockroach).
⑧ CDN & edge delivery
Origin server — your app or object storage (S3)
CDN edge PoPs — cache static assets and cacheable API responses globally
DNS / anycast — route user to nearest edge
Client — receives content from edge, not origin (lower latency, less origin load)
Cache at CDN: static files (JS, CSS, images), video segments, and cacheable GET responses with
Do NOT cache: personalized pages, auth tokens, POST responses, data with
Invalidation: versioned URLs (
Cache-Control headers.Do NOT cache: personalized pages, auth tokens, POST responses, data with
Cache-Control: private/no-store.Invalidation: versioned URLs (
app.v2.js) beat purge APIs. Edge compute (Cloudflare Workers) for redirects and A/B at edge.CDN in request path
User (Tokyo)
User (London)
CDN Edge Tokyo
CDN Edge London
Origin / API (US)
⑨ End-to-end request path
Typical web request path
Client→DNS→CDN→Load Balancer→App Server→Cache→Database
Latency budget example (p99 < 200ms):
- DNS + TLS: 20ms
- CDN hit: 10ms (miss adds origin round-trip)
- LB + routing: 5ms
- App logic: 30ms
- Redis: 1ms
- DB query (indexed): 10ms
- Serialization + network: 20ms
Async off critical path: analytics, emails, search indexing → message queue (Kafka/SQS) → workers. User-facing response returns before side effects complete.
⑩ Revision checklist
- Opened with functional + non-functional requirements
- Calculated QPS, storage, bandwidth with stated assumptions
- Drew CDN → LB → App → Cache → DB diagram
- Explained vertical vs horizontal scaling choice
- Named caching strategy and invalidation approach
- Applied CAP with CP vs AP example relevant to the problem
- Picked SQL vs NoSQL with access-pattern justification
- Mentioned CDN for static assets and cacheable reads
- Identified async processing for non-critical work
- Discussed failure modes: DB down, cache stampede, hot keys