Background Jobs in Node With BullMQ

We’ve covered queues in the abstract. Now the concrete, Node-native version you’ll actually ship: background jobs with BullMQ. BullMQ is a Redis-backed job queue that has become the default for “do this later” work in Node — sending email, generating PDFs, processing uploads, calling slow third-party APIs. This is the second flagship Node lesson; by the end you’ll have a real producer and worker.

Why move work off the request path

Recall the event loop lesson: Node runs your JavaScript on one thread, so a slow handler doesn’t just delay one request — it ties up a worker and degrades everyone. And even async-but-slow work (a 4-second PDF render) means the user stares at a spinner and your HTTP timeouts loom.

The fix is to respond fast and finish later. The request handler does the minimum, enqueues a job, and returns 202 Accepted immediately. A separate worker process picks the job up and does the heavy lifting off the request path entirely.

The API stays snappy and the worker tier scales independently — you can run ten workers for a backlog without touching your API fleet.

The producer: enqueue and return

The producer side lives in your API. You create a Queue and add jobs to it; each job is just a name plus a JSON payload.

import { Queue } from 'bullmq';

const connection = { host: '127.0.0.1', port: 6379 }; // Redis
const emailQueue = new Queue('email', { connection });

app.post('/signup', async (req, res) => {
  const user = await createUser(req.body);

  // Hand off the slow part; don't make the user wait for SMTP.
  await emailQueue.add(
    'welcome',
    { userId: user.id, to: user.email },
    {
      attempts: 5,                                  // retry up to 5 times
      backoff: { type: 'exponential', delay: 1000 }, // 1s, 2s, 4s, 8s...
      removeOnComplete: 1000,                        // cap completed history
      removeOnFail: 5000,
    },
  );

  res.status(202).json({ ok: true }); // fast response, email sends in background
});

The signup feels instant because the only synchronous work is creating the user and dropping a tiny job into Redis. The email itself happens elsewhere.

The worker: process jobs

The worker is a separate process (or set of processes) running a Worker that pulls jobs and runs your async processor. BullMQ handles the locking, acking, and retry bookkeeping; you write the actual job logic.

import { Worker } from 'bullmq';

const connection = { host: '127.0.0.1', port: 6379 };

const worker = new Worker(
  'email',
  async (job) => {
    if (job.name === 'welcome') {
      await sendEmail(job.data.to, 'Welcome!'); // throw to trigger a retry
    }
  },
  {
    connection,
    concurrency: 10, // process up to 10 jobs at once in THIS worker
  },
);

worker.on('completed', (job) => console.log(`${job.id} sent`));
worker.on('failed', (job, err) => console.error(`${job.id} failed: ${err.message}`));

If sendEmail throws, BullMQ catches it and re-queues the job per the attempts and backoff you set on the producer side — no try/catch retry loop needed. After the final attempt it lands in the failed set, where you can inspect or manually retry it (a dead-letter pattern).

Retries with backoff

Transient failures — a flaky SMTP server, a rate-limited API — should be retried, but immediately retrying a struggling dependency makes it worse. Exponential backoff spaces attempts out (1s, 2s, 4s, 8s…), giving the dependency room to recover and protecting you from a retry storm.

Delayed and scheduled jobs

BullMQ jobs don’t have to run now. Two common scheduling needs:

• Delayed jobs — run once, later. “Send a reminder 24 hours after signup”: add the job with { delay: 24 * 60 * 60 * 1000 } and Redis holds it until due.
• Repeatable jobs — run on a cron schedule. “Email a digest every morning at 8am”: add with { repeat: { pattern: '0 8 * * *' } }. BullMQ becomes a distributed cron, no separate scheduler box required.

Concurrency and scaling workers

Two knobs control throughput. Concurrency is how many jobs a single worker runs at once (great for I/O-bound jobs that mostly await). Horizontal workers is how many worker processes you run — start more instances pointed at the same queue and Redis hands each a fair share, no extra config. For CPU-heavy jobs, prefer more processes (or worker_threads) over high concurrency in one process, since CPU work blocks the loop.

You now have a real Node background-jobs system: fast responses, durable retries, schedules, and independent worker scaling. But that last warning is the catch with every queue and worker we’ve discussed — retries mean duplicates. Making duplicates harmless is the final piece: idempotency.