Separate training from inference

Training is the learning phase. Inference is the using phase. Mixing them up leads to wrong expectations about cost, privacy, and what a download actually contains.

During training, software reads batches of data, asks the model to predict an answer, measures the error, and updates the weights. Training also keeps temporary state such as gradients, optimizer values, and the current position in the data. That extra state can be many times larger than the weights alone.

A training system saves checkpoints along the way. A checkpoint lets the run resume after a crash or gives researchers two stages they can compare side by side.

After training finishes, the final weights can be copied into a smaller inference package. Inference does not update the weights for every prompt. It loads them and uses them to calculate an output.

The path looks like this:

training data -> training code -> checkpoints -> released weights
                                                |
                                                v
                                           inference runtime
                                                |
                                                v
                                             response

Downloading released weights does not give you the original dataset, optimizer state, or exact training environment. It gives you enough learned state to run the model, and often enough to fine-tune it further.

Keep this distinction in mind when someone says a model is open. We need to ask which part is open. The weights? The training code? The data documentation?

My advice is to treat inference as read-only by default. If you fine-tune locally, you are starting a small training job again, with its own checkpoints and its own hardware needs.

When you pull a model with Ollama, you are on the right side of this diagram. You receive released weights ready for inference, not a full training stack.