Capabilities and limits

A capable model can explain code, follow examples, transform text, generate programs, inspect images, and reason through unfamiliar problems. This range makes it tempting to think of intelligence as one score.

In practice, model capability is uneven. A model may solve a difficult algorithm and then miss a simple requirement two paragraphs earlier. It may explain an API well but invent one parameter. It may refactor the happy path and forget what happens when the network fails.

Separate knowledge from access

A model may know general facts from training, but that does not mean it knows today’s package version, your production configuration, or what happened in the last five minutes.

Tools can give it current information. A browser can fetch documentation. A shell can run the tests. A database tool can read a record. But tool access does not make the model infallible: it can call the wrong tool, misread the result, or stop too early.

Ask these questions when judging a response:

• Was the needed information in the model context?
• Did the model use the right source or tool?
• Does the conclusion follow from the evidence?
• What important case was not checked?

Confidence is a writing style

Models are trained to produce useful, coherent answers. The words I am certain are generated text, not a measurement from a truth detector.

You can ask a model to state assumptions and uncertainty. This often improves review because it exposes what needs checking. But the model’s confidence still cannot replace an external source, a test, or an observed result.

Match verification to the claim

Different outputs need different checks:

• For a factual claim, read the current primary source.
• For code, run it and test expected and failure cases.
• For a visual change, open the page at relevant sizes.
• For a calculation, recompute a few known examples.
• For a consequential decision, keep a qualified human responsible.

Treat generated output as a candidate. The more expensive an error would be, the stronger the evidence you should require.