Start from the task

Do not start by asking, “What is the best local model?”

Start with the task. A model that summarizes short private notes needs different strengths from one that reads screenshots, writes code, or calls tools across a long workflow.

Write one sentence describing the input and output:

Given one daily activity record, return one factual sentence under 25 words.

That sentence is your contract. Everything else supports it.

Now collect representative examples. Include normal cases, empty input, unusual wording, long input, and content the model must not invent. Ten real examples beat a hundred imaginary ones.

Define what good means. For the sentence above, we can check factual consistency, length, tone, latency, and whether the model adds unsupported details. Write those checks down before you download anything.

Only then compare models. Start with the smallest model that plausibly supports the task. A smaller model is easier to fit, faster to load, and cheaper to operate on your own hardware.

A leaderboard can help you create a shortlist. Your own examples decide which model belongs in the application. I have seen teams ship a 70B model for a job a 3B model handled fine, simply because the bigger name ranked higher on a chart.

When a candidate fails, save the input and the bad output. That failure becomes your next evaluation case. Your test set should grow from real misses, not only from demos that already work.

Try this on your own project: write the one-sentence contract for the feature you want, then list five inputs you already have in production logs or notes. If you cannot find five, you are not ready to pick a model yet.