Architecture, weights, and runtime

A local AI application has several parts. Calling all of them “the model” hides useful differences when something breaks.

The architecture describes the model shape: its layers, attention mechanism, dimensions, and how values move through the network. Two models can share a family name and still differ in layer count or head size.

The weights fill that architecture with learned numbers. Same architecture, different weights, different behavior.

The tokenizer converts text into token IDs the model understands, then converts generated IDs back into text. A mismatch here produces garbage even when the weights are fine.

The runtime loads these files and performs the calculations. Ollama, llama.cpp, MLX LM, and Transformers are examples of runtimes or runtime toolkits.

A model also needs configuration and a chat template. The template turns conversation roles into the exact token pattern used during instruction training. A wrong template can make good weights behave badly. I have seen teams chase “bad model quality” for days when the template was the real bug.

The complete path is:

prompt -> chat template -> tokenizer -> architecture + weights -> token IDs -> text

This explains why one model file does not run inside every application. The runtime must support the architecture, file format, tokenizer, and features you need.

When you evaluate a local setup, list all four layers explicitly. If output looks wrong, check the template and tokenizer before you blame the weights or swap to a bigger model.

Try this on your own project: write the four names on one line for the model you plan to use. If any slot is blank, you are not ready to debug production issues yet.