Run models locally
Choose a local runtime
Compare Ollama, llama.cpp, MLX LM, and desktop interfaces by the job they perform rather than treating them as model families.
A runtime is the software that loads a model and performs inference.
Ollama manages model downloads, local serving, and a simple API. It is a good starting point for this course.
llama.cpp is a lightweight C and C++ inference project with broad hardware support. It runs GGUF models and includes an HTTP server with compatible API routes.
MLX LM targets Apple Silicon. It can generate, quantize, and fine-tune models using Apple’s MLX framework.
Desktop applications such as LM Studio add model discovery and a graphical chat interface. They can be convenient, but the model, format, and runtime compatibility still matter underneath the UI.
Choose based on the boundary you need:
- quick local chat and API: Ollama
- direct GGUF control and broad backends: llama.cpp
- Apple Silicon experiments and fine-tuning: MLX LM
- graphical exploration: a desktop interface
The model is portable only when another runtime supports its architecture, format, template, and features.
Lesson completed