Use dead letters and controlled replay

Some messages will never succeed, no matter how many retries you give them. A payload from a buggy deploy. A reference to a deleted record. A schema version nobody expected. Such a poison message can eat retries forever or block useful work, so the queue needs somewhere to put it.

That somewhere is a dead-letter queue: a normal queue that collects messages after the configured delivery attempts run out:

{
  "queues": {
    "consumers": [
      {
        "queue": "order-jobs",
        "max_retries": 5,
        "dead_letter_queue": "order-jobs-dlq"
      }
    ]
  }
}

After the fifth failed attempt, the message moves to order-jobs-dlq instead of disappearing. Without this setting, exhausted messages are deleted. The failure and its evidence vanish together.

A dead letter is evidence, so treat it that way. Alert on dead-letter growth and log safe failure context: which handler failed, which error class, which jobId. Check the backlog directly:

npx wrangler queues info order-jobs-dlq
# ...
# backlog size: 3

A backlog that was zero yesterday and is three hundred today is telling you a deploy broke something, hours before a customer does.

One more thing. Payloads sit in the DLQ for days. If they carry sensitive data, they need the same retention and access controls as your source data. The dead-letter queue is not exempt from your data rules.

Replay is a deliberate act

The wrong response to a full DLQ is pointing the consumer back at it and hoping. The messages failed for a reason. Unfixed, they’ll fail again.

Do it in order. Fix the code or the data first. Select the messages that the fix addresses. Keep their operation IDs, so idempotency checks still recognize them. Then watch the result.

A small replay consumer on the DLQ does this well:

async queue(batch, env) {
  for (const message of batch.messages) {
    await env.ORDER_JOBS.send(message.body)  // original body, original IDs
    message.ack()
  }
}

Because the original jobId travels with the replayed message, work that partially completed before dying won’t run twice.

Try the loop before you need it: force a schema-version failure, inspect its dead letter, deploy compatible handling, and replay that single message. One rehearsed message teaches you the whole procedure, selection, replay, verification, at a moment when nothing is on fire.