What is RAG? Retrieval before the model answers

Nov 7, 2026

RAG fetches relevant passages and puts them in the prompt so the model answers from those passages. How that differs from search in the coding agents.

RAG (retrieval-augmented generation) means you find the relevant pieces of your own documents first, then put those pieces in the prompt. The model answers from them instead of from memory.

RAG chunking explained owns chunk size, overlap, and what the vectors cost. Embeddings is the list of numbers that makes the “relevant” step a distance. This entry is the short definition, and where the four coding tools do and don’t document that pipeline.

How RAG works

You split documents into passages small enough that one passage is one idea. An embedding model turns each passage into a vector, and you store the vectors. Later you embed the question with the same model and keep the nearest passages.

Those passages go into the context window next to the question. The chat model still has to read them. Retrieval picked them. It did not write the answer.

A grep is the other search. You already know the string. RAG is for when you know the question and not the wording in the document.

RAG in each tool

None of the four coding-agent pages describes the product as a RAG system over your repository. One API next to Grok does.

	Claude Code	Codex	Cursor	Grok
Over your repo	Grep and Glob are in the tools list. No retrieval index is documented	Not documented	Instant Grep, on your machine. The search page says Cursor does not store embeddings of the codebase for search	Not documented in Grok Build
A document collection	Not on the Claude Code docs index	Not on the Codex docs index	Not documented	xAI Collections Search. The page calls it RAG

Claude Code

The tools reference lists Grep and Glob for finding files and patterns, and marks both absent by default on macOS, Linux, and WSL. It does not describe an index of embedded passages. See the tools reference.

OpenAI Codex

The agent guide describes tools that gather information. The docs index does not have a RAG page, a collections page, or a file-search tool for the Codex app and CLI. See Working with agents.

Cursor

Instant Grep is the search Agent uses for an exact match: a function name, an error string, or a regex. The index is built and queried on your machine. The search page says Cursor does not upload file paths or code to build that index, and it does not store embeddings of your codebase for search.

An Explore subagent can run many of those searches and return a summary. That is still the grep index. See Cursor search.

Grok

Grok Build’s overview does not mention retrieval over the repo or a collections tool.

The xAI API does. Collections Search retrieves passages from files you uploaded. The page lists “RAG Applications” as one of its uses, and it says the search is semantic, by meaning, not only by keyword. In the xAI SDK the tool is collections_search. On the OpenAI-compatible Responses API the same tool is file_search. See Collections Search.

Details checked against each vendor’s documentation in November 2026.

A real example

The collections page’s JavaScript sample wires file_search to a collection id and asks what is in it. collectionId here stands for the id that page’s upload step returns. maxNumResults: 5 is the sample’s own cap:

tools: {
  file_search: openai.tools.fileSearch({
    vectorStoreIds: [collectionId],
    maxNumResults: 5,
  }),
}

The sample’s prompt is “What documents do you have access to?” The model searches the collection, then answers from what came back.

For a codebase question, the Cursor search page is the other shape. A function name is a grep, and that page says the agent does that without a stored embedding index. The RAG chunk calculator is the page on this site for chunk count and whether the retrieved text still fits in the window.

When to use RAG

Use it when the answer has to come from a set of documents the model was not trained on, and the set is too big to paste. A handbook, a pile of contracts, or a folder of support notes are the cases the collections page and the chunking post are about.

Skip it when you know the symbol. Cursor’s search page is built for that lookup. Skip it when the whole file fits in the prompt. The chunking post’s reason to cut documents up is that a whole book does not fit, and one vector of a whole book is a blur.

LangChain and LangGraph includes retrieval in the library. The chunking post is still the one that decides the passage size.

Pitfalls

Two vectors are only comparable when the same model produced them. The embeddings entry states the xAI rule: the search API embeds the question with the model that indexed the collection.

A long passage becomes one blurry vector. The chunking post is the walkthrough of size, overlap, and the duplicate tokens overlap makes you store.

Cursor’s documented codebase search is Instant Grep. The search page says Cursor does not store embeddings of the codebase for that search. A function name is a grep.

You still read the passage before the answer quotes it. The collections page can attach citations. The distance by itself is not one of them.

RAG chunking explained is chunk size, overlap, and the cost of storing the vectors, with the RAG chunk calculator.
Embeddings is the vector step, including the OpenAI and xAI embedding calls.
LangChain and LangGraph includes retrieval among the pieces you wire in TypeScript.
What a language model does is the training the model answers from when the passages are not in the prompt.

Embeddings: An embedding is a list of numbers that puts similar text close together. How OpenAI and xAI build them, and why Cursor search says it does not store them.
Context window: The context window is everything a coding agent can see at once. How to check it, free it and keep it small in Claude Code, Codex, Cursor and Grok Build.

All AI glossary entries →

Follow @flaviocopes

Want me to talk about your product? You can sponsor this site.