Cloud or local models

A cloud model runs on the provider’s servers. You send a request over the network and get a response back. A local model runs on hardware you control: your laptop, a workstation, a server in your closet.

Most of the models you hear about are cloud models. But you can download open-weight models and run them for free with tools like Ollama or LM Studio. I do it, and it’s worth trying at least once to see what it feels like.

The choice isn’t powerful versus private. Both sides have tradeoffs.

Cloud models

Easy to start. Strong general capability, big context windows, image input, and someone else handles scaling. You don’t buy hardware and you don’t maintain inference software.

The cost is that your data crosses a boundary. Read the current policy for retention, training, region, and deletion. Consumer chat terms can differ from API or enterprise terms, even at the same company.

You also depend on a network and a vendor. Your feature has to survive latency, outages, rate limits, price changes, and the model being retired.

Local models

A local model works offline and keeps your inputs on the machine. You control the version. For a repeated, high-volume job, owned hardware can make cost predictable.

The honest part: local models are not up to the frontier cloud models for coding. And they need serious hardware. The weights and the context cache live in memory, so a bigger model needs more RAM or GPU memory and produces tokens more slowly on a weak machine. Quantization shrinks a model by storing its weights with less precision. It helps a lot, and it costs some quality.

You also own the maintenance: runtime, model files, updates, security, monitoring. If the model runs on a shared server, you still need authentication and data isolation. Local doesn’t automatically mean secure.

Privacy is about the whole path

Ask where the prompt, files, tool results, output, logs, and backups travel. A local model with a connected web search tool still sends data out. A cloud API can be fine for private work when the contract, configuration, and data minimization match what you need.

Decide which information is allowed into the system first. Then decide where the system runs. Don’t use local as a shortcut around understanding your data.

Mix them

You can route by sensitivity and difficulty. A local model summarizes private notes. A cloud model handles the hard public-code question. A small local model can even redact input before another service sees it.

Whatever you pick, test the exact task on the exact hardware or service. Compare quality, speed, total cost, and how much work it is to operate. The best deployment is the one that meets the requirement, not the one with the nicer label.

If you want to go deeper on running models yourself, the Local AI Models course picks up right here.