Parameters and memory

The weight estimate starts with one formula:

weight GB = parameters in billions x bits per weight / 8

An 8-billion-parameter model stored at 16 bits needs about 16 GB for its weights:

8 x 16 / 8 = 16 GB

At roughly 4 bits, the same estimate becomes 4 GB:

8 x 4 / 8 = 4 GB

Run those numbers before you pull a model. They tell you whether the weights alone fit, not whether the whole session will feel fast.

Real files include scales and metadata. At runtime you also need memory for the context cache, compute buffers, the runtime, the operating system, and other applications. On a 16 GB machine, a 16 GB weight file leaves no room for context or macOS itself.

On Apple Silicon, the CPU and GPU share unified memory. That makes large local models practical, but macOS still needs part of the same pool. On a discrete GPU, model layers can sometimes spill into system RAM, but generation usually becomes slower when they do.

Treat “it fits” and “it runs comfortably” as different claims. Leave headroom, then measure on the real machine.

My advice is to aim for at least 25% free memory after the model loads, then watch Activity Monitor or your GPU tool during a long prompt. Swap churn is the sign you picked too large a model or too long a context window.

If the estimate says 4 GB at Q4 and your laptop has 8 GB RAM, you might still succeed with a short context and a small runtime. If the estimate says 16 GB and you have 16 GB total, pick a smaller model or a lower bit width before you waste an afternoon on failed loads.

Try this on your own project: compute the weight GB for your candidate model at the quantization you plan to use, then compare it to available RAM after you close heavy apps.