Publish/Subscribe and Fan-Out

The last lesson ended on a key distinction: a work queue sends each message to one consumer, but sometimes you want every interested party to react to the same event. That’s publish/subscribe — the pattern behind event-driven architectures, where one “thing happened” ripples out to many independent reactions.

Topics and fan-out

In pub/sub, producers publish messages to a named topic and never address a consumer directly. Consumers subscribe to topics they care about. The broker copies each published message to every subscriber — this copying to many recipients is fan-out.

The publisher of order.placed doesn’t know — and shouldn’t care — that four services react to it. Tomorrow you add a fraud-check subscriber and change zero publisher code. That’s the architectural payoff: producers and consumers evolve independently, wired together only by a topic name.

Decoupling producers from consumers

Pub/sub decouples along three axes at once:

• Space — the publisher doesn’t know who (or how many) consumers exist.
• Time — with a durable topic, a consumer that’s offline catches up later.
• Synchronization — the publisher fires and moves on; it never blocks on consumers finishing.

This is what turns a tangle of direct service-to-service calls into a clean event-driven system. Instead of the order service calling billing, then shipping, then analytics (and breaking when any is slow or down), it emits one event and the rest is somebody else’s subscription.

Push vs pull delivery

How does a message actually get from the topic to a subscriber? Two models:

• Push — the broker delivers messages to subscribers as they arrive, typically by calling an HTTP webhook or invoking a function. Low latency, no polling, but the broker must handle subscriber backpressure and retries, and the subscriber needs a reachable endpoint.
• Pull — subscribers fetch messages on their own schedule (long-poll or read at an offset). The consumer controls its own rate, which makes backpressure natural, at the cost of a bit more latency and client logic.

Pub/sub vs a work queue

This is the comparison interviewers love, because the two look similar but answer different questions.

A work queue is about load balancing a task across a pool — ten workers share the load, each job done once. Pub/sub is about broadcasting a fact — every subscriber gets its own copy and reacts on its own terms.

The two compose neatly: a common pattern is fan-out to per-consumer queues. The topic broadcasts the event, but each subscriber has its own durable queue behind the topic, so within one subscriber a pool of workers load-balances the work. Kafka does this with consumer groups (one copy per group, load-balanced within a group); SNS+SQS does it by subscribing an SQS queue to an SNS topic.

That hybrid gives you the best of both: fan-out across teams, load-balancing within a team. The topic answers “who needs to know,” each queue answers “who does the work.”

The JavaScript angle

Redis pub/sub (which powered our WebSocket fan-out two lessons ago) is the simplest taste of this pattern, and it’s a few lines in Node — though note it’s fire-and-forget with no durability.

import { createClient } from 'redis';

const pub = createClient();
const sub = pub.duplicate();
await Promise.all([pub.connect(), sub.connect()]);

// Subscriber: reacts to every event on the topic.
await sub.subscribe('order.placed', (msg) => {
  const order = JSON.parse(msg);
  console.log('shipping reacts to', order.id);
});

// Publisher: fires the event, knows nothing about subscribers.
await pub.publish('order.placed', JSON.stringify({ id: 'o1' }));

For durable, at-scale pub/sub you’d reach for Kafka topics, NATS, or a cloud service (SNS, Google Pub/Sub) instead. But the mental model is identical: publish to a name, subscribe by name, fan out in between.

We’ve broadcast events and distributed work. Now let’s get concrete about running the work itself in Node — background jobs with BullMQ.