System Design Interview Patterns
Reusable patterns for fan-out, sharding, queues, idempotency, sagas, circuit breakers — and when to apply each.
Interview tip For any scenario, name 2–3 patterns aloud: "fan-out on write for notifications, idempotent consumers, circuit breaker on payment API."
① Pattern map — quick reference
| Pattern | Problem it solves | Watch out for |
|---|---|---|
| Fan-out on write | Precompute feeds/notifications at write time | Celebrity problem — too many followers |
| Fan-out on read | Merge at read time — simpler writes | Slow reads for users following many accounts |
| Sharding | Split data across nodes by key | Cross-shard queries, rebalancing pain |
| Consistent hashing | Minimal key redistribution when nodes change | Hot spots if key distribution skewed |
| Message queue | Decouple producers/consumers, absorb spikes | Ordering, duplicate delivery, poison messages |
| Idempotency | Safe retries without duplicate side effects | Key storage TTL, scope of "same" operation |
| Saga | Multi-step workflow without 2PC locks | Compensating transactions complexity |
| Circuit breaker | Stop calling failing dependency | Half-open state tuning, cascading if misconfigured |
② Fan-out on write vs read
Fan-out on write (push model): When user A posts, push post ID into every follower's feed cache (Redis sorted set). Read = O(1) fetch prebuilt feed. Write = O(followers) — expensive for celebrities.
Fan-out on read (pull model): Store posts by user; at read time merge posts from all followed users. Write = O(1). Read = O(following count) — slow for users following 10K accounts.
Hybrid (Twitter-style)
New post→Fan-out write→<10K followers→Fan-out read→Celebrity accounts
Hybrid: fan-out on write for normal users; celebrities fetched separately at read time. Precompute for active users only.
③ Sharding & consistent hashing
Sharding strategies:
- Hash sharding:
shard = hash(user_id) % N— even spread; changing N requires resharding - Range sharding: user_id 1–1M on shard A — range queries easy; hot ranges possible
- Directory sharding: lookup service maps key → shard — flexible; lookup service is bottleneck
Consistent hashing ring
Key hash→Clockwise to node→Virtual nodes→Even load
Virtual nodes: each physical node owns multiple points on the ring — better load balance. Adding a node only moves ~1/N keys. Used in: Cassandra, DynamoDB, Redis Cluster, CDNs.
④ Message queues & event streaming
| System | Model | Ordering | Retention |
|---|---|---|---|
| RabbitMQ / SQS | Queue — one consumer per message | Single consumer: yes; competing: no guarantee | Until ack / TTL |
| Kafka | Log — multiple consumer groups | Per-partition ordering | Configurable retention (days) |
| Redis Streams | Lightweight log | Per stream | Memory-bound |
Queue decoupling
API Server (producer)
Message Queue / Kafka
Email Worker
Push Worker
Analytics
At-least-once delivery is default — consumers must be idempotent. Dead letter queue (DLQ) for poison messages after N retries. Backpressure: slow consumers → queue depth grows → alert and scale workers.
⑤ Idempotency
Idempotent operation: performing it multiple times has the same effect as once. Critical for retries, network duplicates, and at-least-once queues.
| Technique | How | Example |
|---|---|---|
| Idempotency key | Client sends UUID; server stores result keyed by it | Stripe Idempotency-Key header |
| Natural idempotency | Operation is inherently safe to repeat | PUT /users/123 with full body |
| DB unique constraint | Duplicate insert fails cleanly | UNIQUE(order_id, payment_id) |
| State check | Only act if status is PENDING | Charge only if order.status != PAID |
Store idempotency keys in Redis/DB with TTL (24–72h). Return cached response on duplicate key. Payment APIs: never double-charge — idempotency is non-negotiable.
⑥ Sagas vs two-phase commit
| Approach | Mechanism | Pros | Cons |
|---|---|---|---|
| 2PC | Coordinator locks all participants; commit or abort | Strong atomicity | Blocking, coordinator SPOF, does not scale |
| Choreography saga | Each service emits events; others react | Decoupled, no coordinator | Hard to trace, cyclic risk |
| Orchestration saga | Central saga manager calls steps + compensations | Clear flow, easier debug | Orchestrator is dependency |
Saga: book flight + hotel
Book flight ✓→Book hotel ✗→Compensate: cancel flight
Compensating transactions undo prior steps (cancel reservation, refund). Not all steps are compensatable (email sent). Prefer sagas for microservices; 2PC rare outside databases.
⑦ Circuit breaker
States: Closed (normal) → failures exceed threshold → Open (fail fast, no calls) → after timeout → Half-open (trial request) → success → Closed; failure → Open.
Circuit breaker in call chain
Your Service
Circuit Breaker
Payment API (healthy)
Payment API (down → open)
| Setting | Typical value | Purpose |
|---|---|---|
| Failure threshold | 5 failures in 10s | Trip to open |
| Open duration | 30–60s | Let dependency recover |
| Half-open probes | 1–3 requests | Test recovery |
| Fallback | Cached response / queue for retry | Graceful degradation |
Libraries: Resilience4j (Java), Polly (.NET), Hystrix (legacy). Pair with timeouts — circuit breaker without timeout still hangs threads.
⑧ Rate limiting (companion pattern)
| Algorithm | Behavior | Use case |
|---|---|---|
| Token bucket | Burst allowed up to bucket size; steady refill rate | API rate limits with burst tolerance |
| Leaky bucket | Smooth output rate regardless of input spikes | Traffic shaping |
| Fixed window | Count requests per window (e.g. per minute) | Simple; boundary spike at window edges |
| Sliding window | Rolling count over last N seconds | Smoother than fixed window |
Distributed rate limiting: Redis INCR + EXPIRE or sliding window in Redis (Lua for atomicity). Return
429 Too Many Requests with Retry-After header. Per-user, per-IP, per-API-key tiers.⑨ Scenario — notification system patterns
Write path: event → Kafka topic (partitioned by user_id)
Fan-out workers: read event, lookup preferences, enqueue per-channel jobs
Idempotent consumers: dedupe by (user_id, event_id, channel)
Circuit breaker on Twilio/SendGrid — fallback to in-app only
Rate limit: max 10 push/hour/user to prevent spam
Inbox storage: sharded by user_id, sorted by timestamp
Interview script: "I'd use a queue to decouple the write path from delivery, idempotency keys to handle retries, circuit breakers on external providers, and shard the inbox by user_id."
⑩ Revision checklist
- Named fan-out on write vs read with celebrity/hybrid mitigation
- Explained sharding strategy and consistent hashing for cache/cluster
- Used message queue for async decoupling with at-least-once semantics
- Designed idempotency for payments or duplicate-prone operations
- Contrasted saga vs 2PC for distributed transactions
- Described circuit breaker states and fallback behavior
- Mentioned rate limiting with algorithm choice
- Mapped patterns to a concrete scenario (feed, payments, notifications)