What is Pub/Sub Messaging?
System Design Fundamentals
Publish/Subscribe (Pub/Sub) is an asynchronous service-to-service communication pattern. In this model, any message published to a topic (a logical channel) is immediately received by all subscribers of that topic.
The defining feature is decoupling:
Publishers don’t know who is receiving the messages.
Subscribers don’t need to know who sent the message.
The Message Broker sits in the middle, handling the routing and delivery.
Core Concepts & Filtering
In Pub/Sub, subscribers usually only want a specific subset of messages. This is handled through filtering:
Topic-Based Filtering: Messages are published to named channels (e.g.,
orders,logs). Subscribers receive everything on that channel.Content-Based Filtering: The broker examines the message content and only delivers it if it matches specific criteria defined by the subscriber (e.g., “only messages where
price > 100“).
In-Depth Examples & Use Cases
1. Event Notification (Decoupling Services)
In a traditional system, if a user makes a purchase, the OrderService might have to manually call the EmailService, ShippingService, and InventoryService. If one is down, the whole process might fail.
Pub/Sub Approach: The OrderService simply publishes an “Order Created” message to a topic.
Benefit: The OrderService can “fire and forget”. It doesn’t care if the EmailService is currently offline; the broker will ensure the message is delivered when the service is back online.
Real-World Example: Split.io uses this to send feature flag updates to millions of apps. If an app is offline, it gets the update as soon as it reconnects.
2. Distributed Caching (State Synchronization)
When you have multiple server instances, you need them to have the same data in their local memory (cache).
Pub/Sub Approach: When data changes in the database, a message is published to a “Cache Update” topic. All server instances subscribe to this and update their local cache immediately.
Real-World Example: Seats.io uses this for live seat bookings. When someone buys a ticket, the floor plan is updated across all global caches instantly so two people don’t buy the same seat.
3. Distributed Logging
Managing logs in a distributed system with 100+ servers is difficult if you have to log into each one.
Pub/Sub Approach: Every server publishes its logs to a central “Logs” topic.
Parallel Processing: One subscriber can send those logs to Elasticsearch for searching, while another subscriber monitors them for errors to trigger an SMS alert.
4. Handling “Thundering Herds” (Burstability)
In education or live events, thousands of users might join at the exact same second.
Pub/Sub Approach: A dedicated broker (like Redis or Kafka) is designed to handle massive “fanout” bursts—sending one message to 100,000 people simultaneously.
Real-World Example: Wooclap uses this so students in a large lecture hall can all see a professor’s question on their phones at the exact same time, even on spotty Wi-Fi.