System Design

Design Dropbox / File Storage

Sync, versioning, metadata vs blob split, chunking, delta sync, and multi-device consistency.

Interview tip Metadata in DB, files in object storage. Chunk large files (4MB blocks) with content-hash for dedup and delta sync. Eventual consistency across devices with conflict resolution.

① Functional requirements

Upload, download, delete files and folders
Sync changes across user's devices automatically
File versioning — keep last N versions (default 30 days)
Share files/folders with other users (read/write permissions)
Offline access — sync when back online
Block-level dedup: same content stored once globally
Search files by name (metadata only)

Out of scope (state in interview)

Real-time collaborative editing (Google Docs style)
End-to-end encryption (mention as premium feature)
Desktop full-disk backup
Video streaming from storage

② Non-functional requirements

Sync notification < 5s after change on another device
Upload/download utilizes available bandwidth
99.9% durability — no data loss
Support files up to 50GB
Efficient sync — only changed chunks transferred

③ Back-of-the-envelope scale

Assumptions

500M users, 100B files, avg 500KB → 50 PB logical (before dedup)
Dedup ratio ~3:1 → ~17 PB physical blob storage
10M sync ops/day → ~115/sec; peak 10× during work hours
Chunk size 4MB → avg file = 1–2 chunks; 100B chunks metadata entries
Metadata: 100B files × 500B metadata ≈ 50 TB index
Notification: 115 sync events/sec → long-polling or WebSocket per device

Content-addressable storage: chunk identified by SHA256 hash. Same chunk uploaded by different users stored once. Metadata DB is the bottleneck — shard by user_id. Blob store (S3) is cheap and unlimited.

④ High-level architecture

Dropbox File Sync System

Desktop / Mobile Clients

Sync API + WebSocket Notifications

Upload/Download API

Metadata Service

Block Service (chunking)

Sharing & Permissions

Metadata DB (sharded SQL)

Block Storage (S3, content-addressed)

Notification Service (long-poll/WS)

Version History Store

File change→Chunk + hash→Upload new chunks→Update metadata

Client splits file into 4MB chunks, hashes each (SHA256). Uploads only chunks not already on server (check hash existence). Metadata service records file → chunk list mapping. Sync = exchange metadata diff, then fetch missing chunks.

⑤ Data flow & execution path

End-to-end execution flow

① Client→② API / LB→③ Core services→④ Cache + DB→⑤ Message queue→⑥ Async workers

Sync path: validate → authorize → read/write primary store
Async path: publish domain events → consumers (email, analytics, search index)
Read-heavy path: CDN / edge cache → regional cache → DB replica
Failure path: retry with backoff, DLQ, idempotent handlers

In interviews, trace one user action through this diagram. State what is synchronous (user waits) vs asynchronous (background), and where you enforce idempotency.

⑥ API & interfaces

Endpoint / flow	Purpose	Notes
POST /v1/blocks/check	Check chunk existence	Body: {hashes[]} → {missing[]} — client uploads only missing
PUT /v1/blocks/{hash}	Upload chunk	Content-addressed; idempotent; returns 200 if exists
POST /v1/files	Create/update file metadata	{path, chunks[], parent_folder, device_id} → version
GET /v1/sync/cursor	Get changes since cursor	Returns changed files/folders since last sync cursor
GET /v1/files/{id}/download	Download file	Returns presigned URLs for chunk assembly
POST /v1/shares	Share folder	{folder_id, user_email, permission: read\|write}

⑦ Data model & storage

files: file_id, user_id, path, parent_id, chunk_hashes[], version, modified_at, device_id

blocks: hash (PK SHA256), s3_path, size, ref_count

shares: resource_id, owner_id, grantee_id, permission

Store	What	Why
S3 (content-addressed)	File chunks/blocks	SHA256 key; dedup across all users; 11 nines durability
PostgreSQL (sharded by user_id)	File metadata, folder tree	ACID for metadata; shard at 100M users
Redis	Sync cursors + notifications	Pub/sub per user for change notifications
S3 Glacier	Old versions	Lifecycle policy after 30 days

⑧ Deep dive — core components

Block-level dedup and delta sync

Client computes rolling hash (Rabin fingerprint) for content-defined chunking — boundaries shift minimally on small edits (vs fixed 4MB blocks where any edit changes all subsequent chunks). Each chunk SHA256 checked against server — upload only new chunks. 1MB edit in 1GB file → upload ~1–2 chunks (4–8MB) not 1GB.

Sync protocol

Each device maintains sync cursor (server monotonic version per user). On change: client pushes metadata + new chunks. Server increments user version, notifies other devices via WebSocket. Other devices: GET /sync?since=cursor → list of changed paths → download missing chunks.

Initial sync on new device: full metadata tree download, then parallel chunk download for all files.

Conflict resolution

Two devices edit same file offline: both push on reconnect. Server detects version conflict (expected_version mismatch). Strategy: last-write-wins (simpler) OR conflict copies (Dropbox model — save both as "file (conflicted copy).ext"). User resolves manually.

For folders: rename/move conflicts create duplicate paths. Operational transform not needed — file-level granularity sufficient for sync (not real-time co-editing).

Sharing and permissions

Share grants grantee read/write on folder subtree. Permission check on every API call: traverse parent chain for ACL. Cache ACL in Redis per (user, resource). Shared files reference owner's chunks — no duplication. Revoke: delete share record; async cache invalidation.

⑨ Trade-offs & alternatives

Decision	Option A	Option B	Pick when
Chunking	Fixed 4MB	Content-defined (Rabin)	Content-defined for better delta sync
Conflict	Last-write-wins	Conflict copies	LWW simple; copies safer for users
Sync notify	WebSocket	Long polling	WebSocket lower latency; polling simpler
Dedup scope	Global cross-user	Per-user only	Global saves 60%+ storage; privacy consideration
Versioning	30-day window	Unlimited versions	30-day balances storage; unlimited costly

⑩ 45-minute interview script

0–5 min: Clarify sync, sharing, versioning, offline
5–12 min: Scale — files, storage PB, dedup ratio
12–22 min: Metadata vs blob split architecture
22–30 min: Chunking and content-hash dedup deep dive
30–36 min: Sync protocol with cursors
36–42 min: Conflict resolution strategies
42–45 min: Sharing permissions model

⑪ Likely follow-up questions

Question	Short answer
Two devices edit offline?	Conflict copies saved; user picks winner; or last-write-wins with timestamp
Delete file — chunks garbage collected?	Decrement ref_count on chunks; GC worker deletes ref_count=0 from S3
User deletes account?	Mark metadata deleted; async chunk GC if ref_count hits 0 globally
50GB file upload?	Multipart chunk upload; resume from last confirmed chunk hash
Malware in shared file?	Out of scope; mention async virus scan on upload
Cross-region sync latency?	Multi-region S3 replication; metadata DB primary in user home region

⑫ Revision checklist

Metadata vs blob storage separation
Content-addressed chunks (SHA256)
Delta sync — upload only changed chunks
Sync cursor / changelog protocol
Conflict resolution strategy stated
Block dedup across users
WebSocket notification for sync
File versioning with retention
Sharing ACL model
ref_count for chunk garbage collection

syncdedupobject storagemetadatachunking