Pin and upgrade models

A model name is part of application behavior.

Do not replace it casually because a newer model appears on a leaderboard. The new version may follow prompts differently, use more memory, change tool-call syntax, or regress on your exact task.

Record:

• model repository and revision
• local tag or file hash
• quantization
• runtime and version
• prompt or template version
• generation settings
• evaluation results

Run the saved evaluation before upgrading. Compare quality, latency, memory, and failure cases on the real hardware. A model that wins on general benchmarks can lose on your five critical inputs.

Roll out gradually when the feature matters. Keep the previous artifact available until the new one proves stable. With Ollama, that can mean keeping two tags configured and switching via environment variable.

Weights give you a useful advantage here: you can preserve the exact tested artifact instead of depending on a provider to keep a remote model unchanged. Pin the file, not just the marketing name.

Document the rollback command. If gemma3:1b regresses, you want one step to return to the last known good tag without a fire drill.

My advice is to tie model changes to changelog entries the same way you would for a database migration. Someone on call should know what changed and how to revert it.

Try this on your own project: copy your current model tag into config and add a comment with the date it passed evaluation. That comment is boring until upgrade day, then it saves you.