Moondream 3.1 on Workers AI: fast vision at the edge

Cloudflare partnered with Moondream to bring their latest vision language model to Workers AI. You can now send an image to a Worker and ask questions about it, caption it, or get the coordinates of objects inside it, with responses coming back in well under a second.

The model is @cf/moondream/moondream3.1-9B-A2B.

What is Moondream 3.1?

Moondream is a small, fast vision language model. Version 3.1 uses a mixture-of-experts architecture: 9B total parameters, but only 2B active per request.

That small active set gives you visual reasoning close to much bigger models, with the inference cost and latency of a small one. It also has a 32K token context window, so you can ask detailed, structured questions.

The four skills

The model does four things, selected with a task parameter:

• query lets you ask open-ended questions about an image
• caption generates a short, normal, or long description
• point returns the coordinates of objects matching a phrase
• detect returns bounding boxes for objects matching a phrase

The last two are what make this interesting. Most vision models describe images. Moondream can tell you where things are, which is what you need for robotics, UI automation, and image tooling.

And it’s fast: Cloudflare reports first tokens streaming back in roughly 20–30 ms, with point and detect completing in the tens-to-low-hundreds of milliseconds range for a simple image, and full captions and queries staying under a second.

Using it from a Worker

Same pattern as every Workers AI model: the AI binding and env.AI.run(). Here’s the caption task:

export default {
  async fetch(request, env) {
    const res = await fetch('https://cataas.com/cat')
    const blob = await res.arrayBuffer()

    const response = await env.AI.run('@cf/moondream/moondream3.1-9B-A2B', {
      task: 'caption',
      image: [...new Uint8Array(blob)],
      prompt: 'Generate a caption for this image',
      max_tokens: 512,
    })

    return Response.json({ description: response.description })
  },
}

The image field also accepts a public HTTPS URL or a base64 data URI, so you don’t have to fetch and convert the bytes yourself.

For object detection, switch the task:

const response = await env.AI.run('@cf/moondream/moondream3.1-9B-A2B', {
  task: 'detect',
  image: 'https://example.com/team-photo.jpg',
  prompt: 'face',
})

You get back bounding box coordinates for every match.

What this unlocks

The combination of fast + cheap + edge changes which vision features are practical:

• moderating user uploads before they’re stored, using a caption or query call as a synchronous step in the upload path
• alt text generation for images, at scale
• visual agents that look at screenshots and decide what to click
• live feeds such as camera frames and robotics, where a multi-second round trip to a big model is a dealbreaker

I generate alt text for images on this site with a vision model in a batch script. At these latencies you don’t need the batch because you can do it at request time.

Pricing

Moondream 3.1 costs $0.30 per million input tokens and $1.00 per million output tokens on Workers AI.

Vision inputs tokenize images, so input tokens dominate. A detect call that returns a few coordinates produces almost no output tokens. Point-and-detect workloads are close to free at small scale, and the Workers AI free daily allocation applies here too.

Trying it

Check it’s in the catalog and you’re on a current wrangler:

npx wrangler ai models | grep moondream

Then add the AI binding to your wrangler.jsonc and call it from any Worker:

{
  "ai": {
    "binding": "AI"
  }
}

Small specialized models running at the edge, milliseconds from users, is exactly the direction I expected Workers AI to go. Getting an answer to “where is the button in this screenshot” in under 150 ms makes the model practical for interactive tools.

Want me to talk about your product? You can sponsor this site.