Hallucinations and uncertainty

A model can produce a confident answer that is unsupported or false. This is often called a hallucination or confabulation.

The invention usually fits the surrounding pattern. A method named db.users.findByEmail() sounds reasonable even when the installed database client has no such method.

Common forms

Watch for:

• an invented API, option, package, quote, or citation
• a real API taken from a different version
• a correct general fact applied to the wrong project
• a missing limitation, edge case, or failure path
• a conclusion that is stronger than the supplied evidence

Not every wrong answer comes from missing knowledge. The model may receive the correct file and still misunderstand it. It may call the right tool and draw the wrong conclusion from the result.

Labels are useful, but not evidence

Ask the model to separate observed facts, inferences, and uncertainty. This makes review easier because you can see which claims need checking.

But a model can label a false sentence fact. The label describes how the answer is organized. It does not verify the sentence.

A confidence percentage has the same limitation unless it comes from a tested calibration method for that exact task. 95% confident is still generated text.

Verify according to risk

Check a claim more aggressively when it is:

• unfamiliar or surprising
• version-sensitive
• security or privacy related
• expensive, public, or difficult to reverse
• central to several later decisions

For the invented database method, inspect the installed types or source, read the primary documentation for that version, and run a focused example. Asking the same model are you sure? is not independent evidence.

Retrieval and browsing can reduce missing-information errors, but they do not eliminate hallucinations. The model can retrieve the wrong page, overlook a condition, or misquote the result.

Try this

Take five factual claims from one generated answer. For each, write the evidence that would verify it. If you cannot name evidence, decide whether to remove the claim or report it as unresolved.