Compose Queues and Workflows

Queues and Workflows are good at opposite things. That’s why they work so well together.

A Queue absorbs bursts. Ten thousand requests in a minute become a steady drip of batches, and you decide how many consumers run. A Workflow gives one job durable, multi-step execution. But if you create ten thousand Workflow instances straight from HTTP handlers, you have no buffer and no flow control.

So compose them. The endpoint enqueues a small message and returns fast. The queue smooths the burst. The consumer creates one Workflow instance per complex job:

async queue(batch, env) {
  for (const message of batch.messages) {
    const { jobId, orderId } = message.body

    try {
      await env.ORDER_WORKFLOW.create({ id: jobId, params: { orderId } })
    } catch (e) {
      // instance with this ID already exists: duplicate delivery, safe to ignore
    }

    message.ack()
  }
}

The try/catch is not decoration. Queue delivery is at-least-once, so this consumer will see the same message twice from time to time. The deterministic instance ID is what makes that safe. Creating an instance with an ID that already exists throws instead of starting a second workflow, so the duplicate collapses into a no-op.

Let the platform pick random instance IDs here and every duplicate delivery becomes a duplicate order pipeline.

One ID through the whole system

Use one stable job ID everywhere: HTTP, queue, workflow, D1, logs. The endpoint generates jobId, the queue message carries it, the Workflow instance is named after it, and the database rows reference it:

npx wrangler workflows instances describe order-workflow job_01J2AB3CD4
# status: running
# step "charge payment": complete

When a customer asks about their order, one identifier traces the path through every layer. Break the chain with random workflow IDs or unrelated database keys and every support question becomes an archaeology project.

Don’t orchestrate the trivial

Composition has a cost. More moving parts, more to observe. Single-step work should stay a plain Queue job. A welcome email doesn’t need a Workflow, because the queue’s own retries already cover a single step.

My rule: reserve Workflows for jobs with real sequence. Multiple side effects, waits, or state carried between steps.

Try this on your own: design one endpoint that accepts 100 jobs, buffers them in a queue, and starts at most one Workflow per job ID. Then feed it duplicate submissions and prove the “at most one” holds. That property is the reason this whole composition exists.