Operate local AI responsibly
Pin and upgrade models
Treat a model change like a dependency upgrade with named versions, saved evaluations, staged rollout, and a rollback path.
A model name is part of application behavior.
Do not replace it casually because a newer model appears on a leaderboard. The new version may follow prompts differently, use more memory, change tool-call syntax, or regress on your exact task.
Record:
- model repository and revision
- local tag or file hash
- quantization
- runtime and version
- prompt or template version
- generation settings
- evaluation results
Run the saved evaluation before upgrading. Compare quality, latency, memory, and failure cases on the real hardware.
Roll out gradually when the feature matters. Keep the previous artifact available until the new one proves stable.
Weights give you a useful advantage here: you can preserve the exact tested artifact instead of depending on a provider to keep a remote model unchanged.
Lesson completed