# Background jobs in Node.js with BullMQ

> Build a reliable Node.js background job queue with BullMQ and Redis, including workers, retries, backoff, progress, concurrency, and idempotency.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-07-23 | Topics: [Node.js](https://flaviocopes.com/tags/node/) | Canonical: https://flaviocopes.com/nodejs-background-jobs-bullmq/

Some work should not happen while a person waits for an HTTP response.

Sending an email, resizing an image, generating a PDF, and importing a large CSV file can all take time. They can also fail for reasons outside our application.

If we perform that work inside a request handler, the request becomes slow and fragile.

A background job queue lets the web process describe the work and return quickly. A separate worker performs it.

In this tutorial we'll build an email queue with **BullMQ** and Redis.

We'll add retries, exponential backoff, progress, concurrency, and the most important production feature: making jobs safe to run more than once.

## How the queue fits together

Our system has three parts:

```text
HTTP application -> Redis queue -> worker
```

The application is the **producer**. It adds jobs.

Redis stores job data and queue state.

The worker reserves waiting jobs and processes them.

BullMQ is not a hosted queue. It is a Node.js library that coordinates queue behavior using Redis.

## Start Redis

The easiest local setup uses Docker:

```bash
docker run --name jobs-redis \
  -p 6379:6379 \
  redis:7-alpine
```

Create the Node.js project:

```bash
mkdir bullmq-email-queue
cd bullmq-email-queue
npm init -y
npm install bullmq express
```

Add `"type": "module"` to `package.json`.

## Create the queue

Create `queue.js`:

```js
import { Queue } from 'bullmq'

export const connection = {
  host: process.env.REDIS_HOST || '127.0.0.1',
  port: Number(process.env.REDIS_PORT || 6379),
}

export const emailQueue = new Queue('emails', {
  connection,
  defaultJobOptions: {
    attempts: 5,
    backoff: {
      type: 'exponential',
      delay: 1000,
    },
    removeOnComplete: {
      age: 24 * 60 * 60,
      count: 1000,
    },
    removeOnFail: {
      age: 7 * 24 * 60 * 60,
    },
  },
})
```

The queue name, `emails`, identifies its Redis keys. Producers and workers must use the same name and Redis connection.

The defaults say:

- try a failed job up to five times
- wait longer after each failure
- keep a limited history of completed jobs
- retain failed jobs for seven days

Without cleanup rules, old job records can grow forever.

## Add jobs from an API

Create `server.js`:

```js
import express from 'express'
import { emailQueue } from './queue.js'

const app = express()
app.use(express.json())

app.post('/welcome-email', async (request, response) => {
  const { userId, email } = request.body

  if (!userId || !email) {
    return response.status(400).json({
      error: 'userId and email are required',
    })
  }

  const job = await emailQueue.add(
    'send-welcome-email',
    {
      userId,
      email,
    },
    {
      jobId: `welcome-${userId}`,
    }
  )

  response.status(202).json({
    jobId: job.id,
    status: 'queued',
  })
})

app.listen(3000, () => {
  console.log('API listening on http://localhost:3000')
})
```

Run it:

```bash
node server.js
```

Add a job:

```bash
curl -X POST http://localhost:3000/welcome-email \
  -H 'content-type: application/json' \
  -d '{"userId":"42","email":"me@example.com"}'
```

The API returns `202 Accepted`.

That status is a useful promise: the request was accepted for processing, but the work is not finished yet.

## Create a worker

Create `worker.js`:

```js
import { Worker } from 'bullmq'
import { connection } from './queue.js'

async function sendWelcomeEmail({ email }) {
  console.log(`Sending welcome email to ${email}`)

  await new Promise((resolve) => {
    setTimeout(resolve, 1000)
  })
}

const worker = new Worker(
  'emails',
  async (job) => {
    if (job.name !== 'send-welcome-email') {
      throw new Error(`Unknown job: ${job.name}`)
    }

    await job.updateProgress(10)
    await sendWelcomeEmail(job.data)
    await job.updateProgress(100)

    return {
      sentAt: new Date().toISOString(),
    }
  },
  {
    connection,
    concurrency: 5,
  }
)

worker.on('completed', (job, result) => {
  console.log(`Job ${job.id} completed`, result)
})

worker.on('failed', (job, error) => {
  console.error(`Job ${job?.id} failed`, error.message)
})

const shutdown = async () => {
  await worker.close()
  process.exit(0)
}

process.on('SIGTERM', shutdown)
process.on('SIGINT', shutdown)
```

Run it in another terminal:

```bash
node worker.js
```

The waiting job is processed.

Throwing an error marks the attempt as failed. If attempts remain, BullMQ moves the job back to wait after its backoff delay.

## Why retries need backoff

Imagine an email provider is unavailable for one minute.

Immediate retries hit the same failure repeatedly and add more pressure. Exponential backoff spaces attempts out:

```text
1 second
2 seconds
4 seconds
8 seconds
```

Add a little randomness, called jitter, when a large number of jobs might fail together. Otherwise they can all retry at the same instant.

Retry errors that may disappear:

- network timeouts
- rate limits
- temporary service errors

Do not retry permanent errors forever:

- invalid email addresses
- malformed job data
- missing required configuration

Validate job data before adding it, and classify worker errors when the external service gives enough information.

## Make jobs idempotent

Queues provide **at-least-once processing**.

A worker can finish an external action and crash before Redis records completion. The job may run again.

This means `send welcome email` must be safe to repeat.

The `jobId` in the producer prevents two currently retained jobs with the same ID from being added. It helps, but it is not a complete idempotency strategy. Completed jobs may eventually be removed.

For a durable guarantee, record the business action in your database:

```js
async function sendWelcomeEmail({ userId, email }) {
  const existing = await database.emailDelivery.findUnique({
    where: {
      operationKey: `welcome-${userId}`,
    },
  })

  if (existing) {
    return
  }

  const providerMessageId = await emailProvider.send({
    to: email,
    template: 'welcome',
  })

  await database.emailDelivery.create({
    data: {
      operationKey: `welcome-${userId}`,
      providerMessageId,
    },
  })
}
```

The exact transaction depends on the external service. If the provider supports idempotency keys, send the same operation key there too.

Never design a background job under the assumption that it runs exactly once.

## Report progress and status

Add a status endpoint:

```js
app.get('/jobs/:id', async (request, response) => {
  const job = await emailQueue.getJob(request.params.id)

  if (!job) {
    return response.status(404).json({
      error: 'Job not found',
    })
  }

  response.json({
    id: job.id,
    name: job.name,
    state: await job.getState(),
    progress: job.progress,
    result: job.returnvalue,
    failedReason: job.failedReason,
  })
})
```

This is useful for work that has a visible result, such as a report export.

Do not expose raw job data publicly. It may contain email addresses, file paths, or other private information. Authorize the request and return only the fields the owner needs.

## Listen to queue-wide events

Worker events are local to that worker process.

Use `QueueEvents` when another process needs queue-wide completion or failure events:

```js
import { QueueEvents } from 'bullmq'
import { connection } from './queue.js'

const queueEvents = new QueueEvents('emails', {
  connection,
})

queueEvents.on('completed', ({ jobId }) => {
  console.log(`Job ${jobId} completed`)
})

queueEvents.on('failed', ({ jobId, failedReason }) => {
  console.error(`Job ${jobId} failed: ${failedReason}`)
})
```

BullMQ uses Redis streams for these events, which gives them different behavior from ordinary Redis pub/sub.

Modern BullMQ does not require a `QueueScheduler` for delayed jobs and stalled-job recovery. Old tutorials that add one by default are outdated.

## Concurrency and multiple workers

Our worker uses:

```js
concurrency: 5
```

One process can work on five jobs at once.

This helps I/O-heavy jobs that spend time waiting for an API. It does not make CPU-heavy JavaScript run in parallel on one event loop.

For CPU-heavy image or document processing, use multiple worker processes or worker threads.

You can also run several copies of `worker.js`. BullMQ ensures that one worker reserves a particular job at a time.

Start with low concurrency and measure:

- external API rate limits
- Redis load
- database connections
- memory use
- event-loop delay

More concurrency can make the whole system slower when the database becomes the bottleneck.

## Priorities, delays, and scheduled work

Delay one job:

```js
await emailQueue.add(
  'send-reminder',
  {
    userId: '42',
  },
  {
    delay: 60 * 60 * 1000,
  }
)
```

Give an urgent job higher priority:

```js
await emailQueue.add('send-password-reset', data, {
  priority: 1,
})
```

Lower numeric values have higher priority.

BullMQ also supports repeatable job schedulers. Use those for recurring application jobs, but remember that Redis is then part of the scheduling infrastructure. Monitor it and back it up according to your recovery needs.

## Deploy producers and workers separately

The web application and worker should be separate processes:

```text
web:    node server.js
worker: node worker.js
```

This lets you scale them independently.

The API may need ten instances while two workers are enough. A large import could require twenty temporary workers without changing the web tier.

In production:

- use a persistent Redis service
- enable authentication and TLS
- keep Redis off the public internet
- set memory and eviction policies deliberately
- monitor waiting, active, delayed, and failed counts
- alert on old waiting jobs, not only failed jobs
- shut workers down gracefully during deploys

If Redis evicts queue keys because it ran out of memory, the queue loses state. A job queue is not cache data.

## The important design rule

Moving work to a queue does not make failure disappear.

It gives failure a place to be retried, inspected, and recovered without holding an HTTP request open.

A reliable job should have:

- small, validated input data
- a stable job name
- bounded retries with backoff
- an idempotency strategy
- useful progress or result data
- logs and metrics
- a clear cleanup policy

With those pieces, BullMQ turns Redis into a practical background processing system for Node.js, while keeping slow and failure-prone work away from the request that started it.