Choose a model that fits
Parameters and memory
Estimate the memory required by model weights and leave room for context, runtime buffers, and the operating system.
The weight estimate starts with one formula:
weight GB = parameters in billions x bits per weight / 8An 8-billion-parameter model stored at 16 bits needs about 16 GB for its weights:
8 x 16 / 8 = 16 GBAt roughly 4 bits, the same estimate becomes 4 GB:
8 x 4 / 8 = 4 GBReal files include scales and metadata. At runtime you also need memory for the context cache, compute buffers, the runtime, the operating system, and other applications.
On Apple Silicon, the CPU and GPU share unified memory. That makes large local models practical, but macOS still needs part of the same pool. On a discrete GPU, model layers can sometimes spill into system RAM, but generation usually becomes slower.
Treat “it fits” and “it runs comfortably” as different claims. Leave headroom, then measure on the real machine.
Lesson completed