Stateless Services

Horizontal scaling rests on one principle: any server can handle any request. A request arrives, the load balancer sends it to whichever instance is least busy, that instance does the work, and it never matters which instance it was. A service with that property is stateless — and statelessness is the prerequisite that makes everything in the last three lessons actually work.

“Stateless” doesn’t mean the system has no state. Of course it does — users, carts, sessions all exist. It means the application server doesn’t hold that state in its own process memory. The state lives somewhere shared (a database, a cache, a session store) that every instance can reach equally.

Why in-process state breaks scale-out

Suppose you store logged-in sessions in a plain JavaScript Map inside your app:

const sessions = new Map();   // ❌ lives in THIS process's memory only

On one server this works flawlessly. Add a second server behind a load balancer and it falls apart immediately:

• A user logs in. The request hits Server A, which stores their session in its Map.
• Their next request gets balanced to Server B, whose Map has never heard of them. They appear logged out.

You now have two bad options. Pin the user to Server A with sticky sessions — the band-aid we already condemned, which wrecks load balancing and loses the session when A dies. Or externalize the session so both servers see it. Only the second one actually scales.

The same problem hits any in-process state: an in-memory rate-limit counter counts per-server instead of globally, an in-memory cache has a hit on A and a miss on B, an in-memory feature-flag map drifts between instances. In-process state is invisible to your other instances, and invisibility is fatal to “any server, any request.”

Externalize the state

The fix is mechanical: move shared state out of the process and into a store that all instances connect to — almost always Redis for hot, ephemeral state like sessions, and the database for durable state. Now every server reads and writes the same source of truth, and the load balancer can route freely.

import express from 'express';
import session from 'express-session';
import { RedisStore } from 'connect-redis';
import { Redis } from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);
const app = express();

app.use(session({
  // ✅ session lives in Redis — every instance shares it.
  store: new RedisStore({ client: redis, prefix: 'sess:' }),
  secret: process.env.SESSION_SECRET,
  resave: false,
  saveUninitialized: false,
  cookie: { httpOnly: true, secure: true, maxAge: 86_400_000 },
}));

app.post('/login', (req, res) => {
  req.session.userId = authenticate(req.body);  // written to Redis
  res.json({ ok: true });
});

// This route works no matter which instance the LB picks — the session
// is read back from Redis, not from this process's memory.
app.get('/me', (req, res) => {
  res.json({ userId: req.session.userId ?? null });
});

app.listen(3000);

The change is small but the payoff is total: kill any instance, add ten more, do a rolling deploy — every user’s session survives because it was never tied to a process in the first place. No stickiness required.

The Node traps: WebSockets and in-memory everything

Two patterns lure Node engineers into accidental statefulness.

1. WebSocket connections live in one process. A WebSocket (or socket.io) connection is a long-lived TCP socket held by one specific Node process. If user Alice is connected to Server A and user Bob to Server B, and Alice sends a message to Bob, Server A has no way to reach Bob’s socket — it’s held by B. This is the canonical real-time scaling trap.

The connection itself is unavoidably stateful (you can’t externalize a live TCP socket). The fix is to externalize the message routing: a Redis pub/sub adapter lets every server publish events that all the others receive, so any server can deliver to any connected client.

import { Server } from 'socket.io';
import { createAdapter } from '@socket.io/redis-adapter';
import { Redis } from 'ioredis';

const io = new Server(httpServer);

// Both servers join the same pub/sub fabric. Now io.to(room).emit()
// reaches clients connected to ANY instance — not just this process.
const pub = new Redis(process.env.REDIS_URL);
const sub = pub.duplicate();
io.adapter(createAdapter(pub, sub));

io.on('connection', (socket) => {
  socket.on('chat', (msg) => {
    // Broadcasts cross the Redis bus to every instance's clients.
    io.to(msg.room).emit('chat', msg);
  });
});

2. Module-level variables are per-instance. Any let cache = {}, counter, queue, or Map at module scope is silently per-process. It works in development (one process) and on a single box, then fragments the moment you scale out or even just run cluster. Treat module-level mutable state as a code smell in a service you intend to scale.

Stateless ≠ no state — just shared state

To be precise: a stateless service is one where the instances are interchangeable because all meaningful state lives in shared, external systems. You still have databases, caches, and session stores — they’re just outside the app tier, deliberately, so the app tier can be scaled, replaced, and crashed at will.

Get statelessness right and the whole scaling toolkit clicks into place: load balancing is free, redundancy is real, and deploys are boring. Next we push content even further from the app — all the way to the network edge — with CDNs.