Google SD

Design YouTube

Upload pipeline, transcoding, CDN delivery, adaptive bitrate, and view counting at billions of views.

Interview tip Separate upload path (blob storage + queue + workers) from read path (CDN + HLS/DASH). View counts are approximate — batch aggregate, never sync increment per view.

① Functional requirements

Upload video (resumable, chunked); support up to 12-hour videos
Transcode to multiple resolutions (144p–4K) and formats (H.264, VP9)
Stream playback with adaptive bitrate (HLS/DASH)
Video metadata: title, description, tags, thumbnail
View count display (approximate, eventually consistent)
Search videos by title/tags
Comments on videos (basic; separate service)

Out of scope (state in interview)

Live streaming (RTMP/WebRTC)
Recommendation algorithm / ML ranking
Content ID / copyright detection
Monetization and ads insertion

② Non-functional requirements

Playback start < 2s (time to first frame)
Upload supports resume after network failure
99.9% availability on read (CDN)
Transcoding completes within 2× video duration
Global delivery — low buffering across regions

③ Back-of-the-envelope scale

Assumptions

500 hours video uploaded/min → ~30K uploads/hour → ~8 uploads/sec (avg), 50/sec peak
Avg upload 500MB raw; 5B views/day → ~58K views/sec
Storage: 500hr/min × 60 × 24 × 500MB × 365 ≈ 65 EB/year raw (use aggressive compression)
Transcoding: 8 uploads/sec × 5 renditions × 2 min avg transcode = 80 concurrent transcode jobs
CDN egress: 58K views/sec × 5 Mbps avg = 290 Tbps peak (CDN handles 95%+)
Metadata DB: 8 uploads/sec; 5B view events/day for aggregation

CDN serves 95%+ of video bytes — origin only on cache miss. View counts aggregated in memory (per-video counter shard) and flushed to DB every 10–60 seconds. Exact counts not required for display.

④ High-level architecture

YouTube Video Platform

Upload Clients

Playback Clients

Upload API (chunked)

Video API (metadata)

Object Storage (S3 raw)

Transcode Queue + Workers

Transcoded Storage + CDN

Metadata DB

Search Index

View Counter Service (Redis)

Analytics Pipeline

Chunk upload→S3 assemble→Transcode→CDN publish

Upload and playback are completely separate paths. Client uploads chunks to S3 via presigned URLs; API tracks upload session state. Transcoding is async — video unavailable until all renditions ready (or publish low-res first).

⑤ Data flow & execution path

End-to-end execution flow

① Client→② API / LB→③ Core services→④ Cache + DB→⑤ Message queue→⑥ Async workers

Sync path: validate → authorize → read/write primary store
Async path: publish domain events → consumers (email, analytics, search index)
Read-heavy path: CDN / edge cache → regional cache → DB replica
Failure path: retry with backoff, DLQ, idempotent handlers

In interviews, trace one user action through this diagram. State what is synchronous (user waits) vs asynchronous (background), and where you enforce idempotency.

⑥ API & interfaces

Endpoint / flow	Purpose	Notes
POST /v1/uploads/init	Start upload	Returns uploadId, presigned chunk URLs; {filename, size, checksum}
PUT /v1/uploads/{id}/chunks/{n}	Upload chunk	Direct to S3 multipart; API tracks completed parts
POST /v1/uploads/{id}/complete	Finalize upload	Triggers transcode job; returns videoId when processing
GET /v1/videos/{id}/manifest.m3u8	Playback	HLS manifest listing rendition URLs on CDN
POST /v1/videos/{id}/view	Record view	Async; dedupe by user/session per 24h; increments counter
GET /v1/videos/search?q=	Search	Elasticsearch; ranked by relevance + view count

⑦ Data model & storage

videos: video_id, user_id, title, description, status (uploading|processing|ready), view_count, created_at

renditions: video_id, resolution, codec, cdn_url, bitrate

upload_sessions: upload_id, completed_parts[], expires_at

Store	What	Why
S3 / GCS	Raw + transcoded video blobs	Multipart upload; lifecycle to Glacier for old raw
CDN (CloudFront/Akamai)	Video delivery	Edge cache; 95%+ hit ratio; origin shield
PostgreSQL	Video metadata	Sharded by video_id; read replicas for search hydration
Redis	Hot view counters	INCR per video; flush batch to DB; shard by video_id
Elasticsearch	Video search index	Title, tags, description; updated on publish

⑧ Deep dive — core components

Resumable chunked upload

Client requests upload session → receives video_id + presigned S3 multipart URLs for 5MB chunks. Uploads chunks in parallel (4–8 concurrent). On failure, resume from last completed part (tracked in upload_sessions DB). Complete call assembles S3 multipart and enqueues transcode.

Transcoding pipeline

Kafka job: {video_id, s3_raw_path, renditions: [360p, 720p, 1080p]}. Worker pool (FFmpeg on GPU instances) produces H.264 segments. Output to S3 transcoded bucket; CDN invalidation/warm. Publish low-res (360p) first for faster time-to-view; upgrade manifest as higher renditions complete.

Priority queue: premium creators transcoded first. Auto-scale workers on queue depth. Dead letter for corrupt uploads.

Adaptive bitrate streaming (ABR)

HLS: video split into 6-sec .ts segments per rendition. Client downloads .m3u8 manifest listing all renditions. Player monitors buffer + bandwidth; switches rendition without rebuffering. DASH is similar with .mp4 segments.

CDN caches segments at edge — same segment requested by millions. Origin only on cold start. For viral video: proactive CDN prefetch to top POPs.

View count at scale

Never sync DB increment per view. Client beacons view event → API → Redis INCR video:{id}:views. Dedupe: SET video:{id}:viewers:{user_id} NX EX 86400. Flush aggregator every 30s: batch UPDATE videos SET view_count += delta. Display "~1.2M views" with rounding.

⑨ Trade-offs & alternatives

Decision	Option A	Option B	Pick when
Upload	Direct S3 presigned	Through API server	Presigned offloads bandwidth from API
Transcode	Sync (wait)	Async queue	Async mandatory; sync blocks user minutes
Streaming	HLS	Progressive MP4	HLS for ABR; progressive only for short clips
View count	Approximate batch	Exact real-time	Approximate standard; exact too expensive
Storage	Delete raw after transcode	Keep raw forever	Delete saves 50%+ storage cost

⑩ 45-minute interview script

0–5 min: Clarify upload, playback, view counts, search
5–12 min: Scale — upload rate, views/day, CDN egress
12–22 min: Diagram upload vs playback paths separately
22–30 min: Chunked resumable upload deep dive
30–38 min: Transcoding pipeline and ABR/HLS
38–42 min: View count aggregation strategy
42–45 min: Viral video CDN prefetch

⑪ Likely follow-up questions

Question	Short answer
4GB upload fails at 90%?	Resume from part list in upload_sessions; S3 multipart complete with existing parts
Transcode backlog during viral upload spike?	Auto-scale GPU workers; priority queue; publish 360p first
View bot inflation?	Dedupe per user/session; CAPTCHA on suspicious patterns; rate limit view API
Copyright on upload?	Out of scope but mention Content ID fingerprint pipeline async
Global users, one region S3?	Multi-region S3 replication; CDN has global POPs; transcode near upload region
Thumbnail generation?	Extract frame at 10% duration during transcode; store in CDN; separate from video pipeline

⑫ Revision checklist

Separate upload and playback paths
Resumable chunked upload to S3
Async transcode queue (FFmpeg workers)
Multiple renditions (360p–1080p)
HLS/DASH adaptive bitrate explained
CDN for 95%+ video delivery
Approximate view count via Redis batch
View dedupe per user per 24h
Metadata DB separate from blob storage
Publish low-res first for faster availability

CDNvideotranscodingHLSobject storage