Query, filter, and evaluate retrieval

Querying is the ingest path in reverse. Embed the question with the same model, then ask Vectorize for the nearest vectors:

const { data } = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: 'how do I cancel a streaming response?',
})

const matches = await env.VECTORS.query(data[0], {
  topK: 5,
  filter: { tenant: 'acme' },
  returnMetadata: 'indexed',
})

The same-model requirement is absolute. Embed the query with a different model than the documents and you get results that look plausible and mean nothing, because the two vector spaces are unrelated.

Filter before ranking

The filter above narrows the candidates first. Then the five nearest neighbors are picked from inside that set. Use namespaces or indexed metadata this way to keep one tenant from ever retrieving another tenant’s documents.

The order is a security property, not an optimization. Query first and filter the matches afterward and you built the leak yourself. Tenant B’s document was the nearest neighbor, got selected, then got dropped. On a different day, with a different question, it survives into the prompt. Filtering after top-k also silently shrinks your results: you asked for five and got two.

Return only the metadata you need, and load the source text from its authoritative store. The index holds pointers. D1 or R2 holds the text. That read is also where authorization happens. Check that the requesting user may see the document before its content goes anywhere near a prompt.

Measure retrieval alone

Similarity is not correctness. A cosine score of 0.82 tells you two vectors are close. It does not tell you the chunk answers the question.

So build a small labeled set and measure retrieval before you add generation:

const cases = [
  { question: 'how do I cancel a streaming response?', expect: 'lesson:cloudflare-ai/streaming#2' },
  { question: 'which metric does the index use?', expect: 'lesson:cloudflare-ai/indexes#1' },
]

Run each question, check whether the expected ID lands in the top five, and record the hit rate. Do this now, because retrieval failures become invisible once generation is added. The model fills the gaps with fluent guesses, and the answer looks fine until someone checks it.

Test ten questions, record hits and misses, then change chunking or filters based on what you saw. One variable at a time: re-chunk, re-run the ten, compare. That loop is the whole discipline of retrieval work.