Design Twitter / a Social Media News Feed
HardAsked at: Meta, Twitter/X, LinkedIn, Amazon
Users follow other users, post short messages ('tweets'), and see a reverse-chronological (or ranked) feed aggregating tweets from everyone they follow. The core challenge is the fan-out problem: efficiently generating feeds for users who follow thousands of accounts, and for celebrities with hundreds of millions of followers.
Request flow
Functional Requirements
- Post a tweet (text + optional media).
- Follow/unfollow other users.
- View a home timeline: recent tweets from everyone you follow, newest first.
- Like/retweet/reply (secondary features, mention only).
Non-Functional Requirements
- Timeline reads must be very fast (sub-200ms) — this is the most frequent, most latency-sensitive operation in the whole system.
- System must tolerate a hugely skewed follower graph (most users have few followers; a small number have 100M+).
- Eventual consistency is acceptable — a tweet appearing a few seconds late in followers' feeds is fine.
- High write availability: tweeting should basically never fail even under load.
Capacity Estimation
- Assume 300M daily active users, each viewing their timeline ~5x/day → 1.5B timeline reads/day ≈ ~17,000 reads/sec average (much higher at peak).
- Assume 5M tweets/day written ≈ ~60 writes/sec average — reads dwarf writes by orders of magnitude, which strongly favors precomputing feeds on write (fan-out-on-write) over computing them on read.
- Average follower count might be ~200, but the distribution is a power law — a handful of accounts have 100M+ followers, which is the crux of the hard part of this problem.
Design Walkthrough
1. The core design decision — fan-out on write vs. fan-out on read
Fan-out-on-write (push model): when a user tweets, immediately write that tweet into the precomputed timeline (a list in a cache/store) of every follower. Reads become a single fast lookup of the requester's own precomputed timeline. Great for the common case (few followers) but catastrophic for celebrities — one tweet from a 100M-follower account would mean 100M writes.
Fan-out-on-read (pull model): store tweets once; when a user requests their timeline, fetch recent tweets from everyone they follow and merge them on the fly. Writes are cheap (O(1)), but reads become expensive (must fan out to potentially thousands of followees at read time) — bad for the common case since reads vastly outnumber writes.
The interview-favorite answer is a hybrid: fan-out-on-write for the vast majority of users (fast reads, and writes are cheap because follower counts are small), but fan-out-on-read (or a hybrid merge) specifically for celebrity/high-follower accounts — a follower's timeline is assembled by merging their precomputed feed (from normal followees) with a live, on-demand fetch of tweets from the small set of celebrities they follow.
2. High-level architecture
Tweet write path: Client → API → Tweet Service writes the tweet to a Tweet DB (source of truth) → publishes a 'tweet_created' event to a message queue → Fan-out Service consumes it, looks up the author's followers (from a Graph/Follower service), and pushes the tweet ID into each follower's precomputed timeline (stored in a fast store like Redis, as a capped list, e.g. most recent ~800 tweet IDs).
Celebrity exception: the Fan-out Service checks follower count; above a threshold, it skips fan-out entirely and just relies on read-time merging.
Timeline read path: API fetches the requester's precomputed timeline list (fast, from Redis) → merges in a live fetch of the small number of celebrities they follow → hydrates the tweet IDs into full tweet objects (batch-fetched from a Tweet cache/DB) → returns.
3. Data storage
Tweet DB: append-only, huge write-once/read-many table {tweet_id, author_id, text, media_url, created_at} — sharded by tweet_id (or author_id, trading off between write locality and even distribution).
Follower Graph: a dedicated store (graph DB or a sharded adjacency-list table {user_id, follower_id}) supporting 'get all followers of X' and 'get all followees of X' efficiently — this is its own interesting sub-problem given some graphs have hundreds of millions of edges.
Precomputed timelines: stored as a capped list of tweet IDs per user in Redis, not full tweet content — keeps the fan-out writes small and avoids duplicating tweet content across millions of followers' timelines.
4. Ranking (beyond simple reverse-chronological)
A production feed isn't just reverse-chronological — a ranking service scores candidate tweets (recency, engagement predictions, author affinity) and reorders the merged candidate set before returning it. This is typically a separate, swappable stage after the fan-out/merge step, often ML-driven, and can be treated as a black box in an interview unless the interviewer wants to go deep on ML systems.
Bottlenecks & Mitigations
- Celebrity fan-out is the single biggest bottleneck — solved by the read/write hybrid approach above.
- Follower-graph reads for fan-out ('who follows this author') at write time must be fast and paginated for huge accounts — process fan-out in batches asynchronously via the queue, not synchronously in the tweet-post request path.
- Precomputed timeline caches can grow large in aggregate memory across hundreds of millions of users — cap each list length (e.g. 800 most recent tweet IDs) and evict/rebuild from source-of-truth on cache miss.
Likely Follow-Up Questions
- How would you handle a user unfollowing/blocking someone — do you retroactively remove their tweets from precomputed timelines? (Usually not worth the cost; filter at read time instead, or accept brief staleness.)
- How would you support 'edit tweet'? (Precomputed timelines store tweet IDs not content, so an edit just updates the Tweet DB row — no fan-out changes needed, a nice side benefit of the ID-only precomputed list design.)
- How do you avoid showing duplicate/near-duplicate content (retweets of retweets) in a timeline? (Dedupe by original tweet_id during the merge/ranking stage.)