Weights are learned numbers

A language model holds a huge collection of numbers called parameters. Most of those parameters are weights.

Think of the model as an enormous audio mixing desk. Its architecture defines how the desk is wired. The weights are the positions of billions of small knobs.

Before training, those values are mostly random. The model cannot produce useful language yet. Ask it a question and you get nonsense.

During training, the model receives examples and tries to predict what comes next. Each error causes tiny adjustments to the weights. Repeat that across a large dataset and the knobs settle into useful positions.

The final weights do not contain a list of complete answers. They encode patterns that influence how an input is transformed and which token becomes likely next.

This is why a model can combine ideas in a way that does not match one training document. It is also why the model can invent a plausible answer. The weights produce likely continuations, not a database lookup with guaranteed facts.

When a model is called 1B, it has roughly one billion parameters. A 30B model has roughly 30 billion. You can sanity-check the memory footprint with a quick estimate:

weight GB ≈ parameters in billions × bits per weight / 8

An 8B model at 16 bits per weight needs about 16 GB just for the numbers on disk. Parameter count affects memory and compute, but it does not tell you whether the model is good at your task.

Notice the difference between memorization and generalization. A lookup table would return the same training sentence every time. Weights let the model guess the next token from patterns, which is powerful and also risky when facts matter.

The weights are the model’s learned state. They are the result of training, not the training process itself.

Try this on your own project: pick one model card and write down its parameter count, quantization, and intended use. That single check often tells you whether the model even fits your machine before you download it.