Design a Chat/Messaging App (WhatsApp-style)
MediumAsked at: Meta/WhatsApp, Amazon, Microsoft
One-to-one and group real-time messaging with delivery/read receipts and offline message delivery. The core challenges are real-time delivery to online users, reliable storage/delivery for offline users, and message ordering.
Request flow
Functional Requirements
- Send/receive text messages 1:1 and in groups, in real time when both parties are online.
- Deliver missed messages once an offline user comes back online.
- Delivery and read receipts (sent, delivered, read).
- Message history retrieval when opening a conversation.
Non-Functional Requirements
- Low latency delivery for online users (sub-second, feels 'instant').
- No message loss, even across disconnects/app restarts/server failures.
- Must scale to hundreds of millions of concurrently connected users.
- Messages should arrive in the order they were sent, per conversation.
Capacity Estimation
- 500M daily active users, sending ~40 messages/day each → 20B messages/day ≈ ~230,000 messages/sec average (design for multi-x peak, e.g. New Year's Eve).
- A message is small (~1KB with metadata) → 20B * 1KB/day ≈ 20TB/day of new message data — very shardable, and often has a retention/archival policy after delivery confirmation.
- Hundreds of millions of concurrent WebSocket connections need to be held across the fleet — this connection-holding capacity, not raw message throughput, is usually the harder scaling axis.
Design Walkthrough
1. High-level architecture
Every client holds a persistent WebSocket connection to one of many stateless-ish Connection/Gateway servers (stateless in that any server can serve any user, but each holds live connection state for whoever is currently attached to it).
A Session/Presence registry (Redis) maps `user_id -> connection_server_id` (and marks users online/offline) so any server can figure out where to route a message for a given recipient.
Send path: Client A sends a message over its socket to Gateway A → Gateway A calls the Message Service, which persists the message durably (Message Store) and looks up where recipient B is connected (Session registry) → if B is online, publish the message to B's gateway via pub/sub for immediate push; if B is offline, it's already durably stored and will be delivered on reconnect.
2. Guaranteed delivery & offline messages
Persist every message to a durable Message Store the moment it's received — before attempting real-time delivery — so a message is never lost even if the recipient is offline or a gateway crashes mid-delivery.
On reconnect, the client sends the ID/timestamp of the last message it has seen; the server queries the Message Store for anything newer addressed to that user and delivers the backlog — this makes offline delivery just a special case of normal history sync, not a separate system.
Use per-conversation monotonic sequence numbers (not wall-clock time, which can skew across servers) so ordering is well-defined and gaps are detectable by the client.
3. Data storage
Message Store: a wide-column / KV store (Cassandra/DynamoDB/HBase) is a great fit — write-heavy, keyed by conversation_id with messages ordered by sequence number, naturally partitioned by conversation, and doesn't need complex joins.
Group chat fan-out: for small/medium groups, write the message once and fan out delivery notifications to each online member's gateway (cheap since group sizes are bounded, unlike Twitter's celebrity-follower problem) — no need for the fan-out-on-write-vs-read tradeoff at this scale.
4. Receipts & presence
Delivery/read receipts are just additional small events flowing through the same real-time pipe (sender's gateway receives a 'read' event for message X from B, forwards it to A's gateway if A is online, and updates the persisted message status either way).
Presence (online/last-seen) is inherently approximate/eventually-consistent — update it on connect/disconnect and periodic heartbeats, and accept brief staleness (e.g. a few seconds delay showing someone went offline) as an acceptable tradeoff for not needing strong consistency here.
Bottlenecks & Mitigations
- Holding hundreds of millions of concurrent connections — horizontally scale the Gateway tier and keep per-connection memory minimal; this is fundamentally a memory/FD scaling problem, not a CPU one.
- Routing a message to 'wherever the recipient is currently connected' across a large gateway fleet — solved by the Redis-backed session registry + pub/sub push described above.
- Group messages to very large groups (thousands of members) can still create a mini fan-out spike — cap group sizes or apply the same read/write hybrid idea from the Twitter case study for unusually large groups/broadcast lists.
Likely Follow-Up Questions
- How would you support end-to-end encryption without the server being able to read message content? (Clients hold keys; server only ever stores/relays ciphertext — this changes very little about the architecture above, mainly the payload becomes opaque bytes.)
- How do you sync message history across a user's multiple devices? (Same offline-catch-up mechanism, applied per-device rather than per-user — each device tracks its own 'last seen sequence number'.)
- How would you detect and handle a gateway server crashing with connections attached? (Clients detect the dropped socket and reconnect to a new gateway via the LB; the session registry entry is cleaned up via TTL/heartbeat expiry so messages aren't misrouted to the dead server.)