# A deep dive into open-weight AI models > Learn what AI model weights are, what open-weight releases let you do, how local inference works, and why open weights are not always open source. Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-08-11 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/open-weight-models/ Meta released a new AI model called [Muse Glimmer](https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model) in August 2026. It is a 30-billion-parameter model designed for local AI agents. It can read text and images, call tools, and recover when a tool fails. But the part that caught my attention was this: Meta released it as an **open-weight model**. The full model would normally need more than 55 GB of memory. Meta also released quantized versions that fit in less than 20 GB. This means you can download one of those versions and run it on your own computer. No request has to travel to Meta's servers. That sounds a little like open source software. It is related, but it is not the same thing. To understand the difference, we first need to understand what weights are. ## What are model weights? Think of a large language model as a huge audio mixing desk. The desk has billions of small knobs. Each knob changes the signal a little as it passes through. Before training, those knobs contain mostly random values. The model is not useful yet. During training, the model sees text, images, code, or other data. It tries to predict an answer, measures how wrong it was, and adjusts the knobs. It repeats this process many times. Those learned knob values are the **weights**. A real model does not store knobs, of course. It stores billions of numbers inside tensors, which are multi-dimensional arrays of numbers. When you see a model described as `8B` or `30B`, the `B` means billion parameters. Most of those parameters are weights. Muse Glimmer has about 30 billion of them. The weights are not the code that trains the model. They are the result of training. You can think of them as the model's learned state. ## The architecture and the weights are different things The **architecture** describes how the model is built. It defines the layers, connections, attention mechanism, and the path data follows through the model. The **weights** are the learned numbers placed inside that architecture. Two models can use a similar architecture but have very different weights. They will behave differently because they learned from different data or were trained for different goals. The runtime is another separate piece. It is the software that loads the architecture and weights, then performs the calculations needed to generate an answer. So when you run a local model, you need several parts: - the model architecture and configuration - the weights - a tokenizer that turns text into tokens - an inference runtime such as Ollama, llama.cpp, MLX, or Transformers - enough memory and compute to run everything An API hides all of this behind an HTTP request. An open-weight release lets you bring those pieces onto your machine. ## What does open-weight mean? An **open-weight model** is a model whose learned weights are available to download. The exact rights depend on the license. Depending on that license, you may be allowed to: - run the model on your own hardware - keep a fixed copy of a specific version - fine-tune it for a task - quantize or convert it to another format - inspect and research its behavior - redistribute the original or modified weights - use it in a commercial product Do not assume all of those rights are included. Some model licenses restrict commercial use, the number of users, certain industries, or how modified versions can be shared. Read the model card and the license before building on a model. Muse Glimmer is a useful example because Meta released its artifacts under the permissive Apache 2.0 license. Its [model card](https://huggingface.co/meta-models/Muse-Glimmer-30B) includes full-precision weights, two 4-bit versions, the vision encoder, and a small companion model used to speed up generation. ## Open weight does not always mean open source People often call downloadable models "open source models". Sometimes that is accurate. Sometimes only the weights are open. With normal software, source code is the preferred form for studying and changing a program. A compiled application is useful, but it does not tell you everything about how the program was created. Model weights have a similar problem. You can run and modify them. But the weights alone do not tell you exactly which data produced them, how that data was filtered, or which training process was used. The [Open Source AI Definition](https://opensource.org/ai/open-source-ai-definition) from the Open Source Initiative asks for more than downloadable weights. It includes the code and detailed data information needed to study and modify the system, along with the model parameters. This gives us a useful distinction: > Open weights tell you that you can get the learned parameters. Open source AI should also give you the materials and freedoms needed to study, modify, and share the wider system. The boundary is still debated. But "open weight" is the more precise term when a company releases the trained parameters without everything used to create them. ## What happens when you download a local model? The model repository normally contains one or more very large files. [Files in the `safetensors` format](https://huggingface.co/docs/safetensors/main/index) hold tensors and nothing executable. This makes the format safer and faster to load than older Python pickle-based model files. For local inference, you will also see `GGUF` files. [GGUF](https://huggingface.co/docs/hub/gguf) stores the model tensors together with metadata in one file. Runtimes including llama.cpp, Ollama, and LM Studio can load it. The file may have a name like this: ```text model-Q4_K_M.gguf ``` The `Q4` part tells you the weights were quantized to around 4 bits. The rest identifies the quantization method. The runtime reads the configuration, allocates memory, loads the weights, and waits for a prompt. When you enter text, the tokenizer converts it into token IDs. The model passes those tokens through its layers and uses the weights to calculate probabilities for the next token. It selects a token, adds it to the sequence, and repeats. Nothing has to call a cloud API. The calculations happen on your hardware. If you want to try that process, the free [Local AI Models Course](https://flaviocopes.com/courses/local-ai-models/) walks through installing Ollama, downloading a model, and talking to it through the local API. ## Why quantization matters Weights take a lot of memory. A 30-billion-parameter model stored with 16 bits per weight needs about 60 GB just for the raw weight values: ```text 30 billion × 16 bits ÷ 8 = 60 GB ``` The exact file and memory size varies with the model and format. But this estimate tells us why full-precision models are difficult to run at home. **Quantization** stores the weights using fewer bits. At roughly 4 bits per weight, the same estimate becomes: ```text 30 billion × 4 bits ÷ 8 = 15 GB ``` Real files also need metadata, scales, and sometimes other model components. Meta's compact Muse Glimmer release is around 17 GB, with the full local setup targeting computers with 24 GB or 32 GB of memory. Quantization is a tradeoff. A smaller file uses less memory and may run faster, but reducing precision can change model quality. Good quantization methods preserve much more quality than the simple bit count suggests. Still, you should test the exact model and quantization on your task. I explained the complete memory calculation in [how much VRAM you need to run an LLM locally](https://flaviocopes.com/llm-vram-requirements/). ## Why open weights matter The obvious benefit is local execution. But open weights change more than where a prompt runs. ### You control the model version A cloud provider can update or remove a model. With downloadable weights, you can keep the exact version you tested. Your application does not change because a provider silently replaced the model behind an API name. This is useful for repeatable tests, research, and products that need stable behavior. ### Private data can stay on your machine A local model can process notes, source code, or documents without sending the prompt to a model provider. But local does not automatically mean private. An agent can still call a search API, upload a file, write logs, or send tool results elsewhere. You must inspect the complete data flow, not only the model. ### You can change the model Open weights let researchers and developers fine-tune, merge, quantize, and study models. A community can make one release work across Apple Silicon, NVIDIA GPUs, CPUs, phones, and embedded devices. It can also find problems that the original creator missed. ### You are not tied to one service The same model may run through several runtimes and hosting providers. You can start locally, move it to your own server, or use a hosted inference service when traffic grows. The model becomes portable in a way a closed API is not. ### Small models become building blocks Not every task needs the strongest cloud model. A smaller open-weight model can classify messages, extract fields, summarize private documents, or handle one step inside a larger system. This is where local models are most interesting to me. They do not need to win every benchmark. They need to perform one useful task reliably enough. ## What open weights do not give you Open weights are not magic. They do not guarantee: - strong output quality - accurate answers - unbiased training data - safe tool use - low hardware costs - fast generation on your computer - permission to use the model however you want - enough information to reproduce the training run You also become the operator. You choose the runtime, secure the machine, install updates, measure quality, and decide what the model can access. For an agent that can change files or call external services, running locally does not remove the need for permissions and confirmation. ## How I would use an open-weight model I would not replace every cloud model with a local one. I would start with small, frequent tasks where privacy and predictable costs matter more than frontier reasoning. For example, I recently added an optional local LLM to a Swift app. The model turns factual activity records into one short daily sentence. If Ollama is missing or the model fails, the original data still works. That is the kind of boundary I like. The model adds value, but the product does not depend on it. For a model like Muse Glimmer, I would also experiment with a private local agent. It could inspect files, interpret screenshots, and call a small set of approved tools. I would keep destructive actions behind confirmation. I would also compare its results against a strong cloud model before trusting it with long coding tasks. I would not use a local model when my laptop cannot run it comfortably, when a small number of API calls costs less than the hardware, or when the task needs the best available reasoning. Local is an option, not a religion. ## How to evaluate an open-weight model When you find a model on Hugging Face, check these things before downloading it: 1. **License.** Can you use and modify it for your project? 2. **Parameter count.** Is the model large enough for the task and small enough for your machine? 3. **Quantization.** Which versions are available, and how much memory do they need? 4. **Runtime support.** Does it work with Ollama, llama.cpp, MLX, Transformers, or your chosen server? 5. **Context length.** Can it hold the input your task needs, and how much extra memory will that context use? 6. **Model card.** What was it designed for, how was it evaluated, and what limitations does the creator list? 7. **Your own test set.** Does it work on real examples from your application? Do not choose a model from a leaderboard alone. The best model is the smallest one that performs your task well enough on hardware you can operate. If you are deciding between local hardware and an API, I also built a [local LLM vs API cost calculator](https://flaviocopes.com/tools/local-vs-api/) and wrote out [the complete cost comparison](https://flaviocopes.com/local-llm-vs-api-cost/). ## The part worth remembering A model is an architecture filled with learned numbers. Those numbers are the weights. When the weights are available under a license that lets you use them, you can move the model out of a company's API and onto hardware you control. That gives you local execution, stable versions, more room to experiment, and less dependence on one provider. But open weights are only one part of an AI system. They do not automatically reveal the training data, reproduce the training process, make the model safe, or turn it into open source AI. That distinction matters. And as capable models start fitting on ordinary computers, it will matter even more. If you want to put this into practice, my free [Local AI Models Course](https://flaviocopes.com/courses/local-ai-models/) takes you from open weights to a working local AI feature with Ollama and Node.js. For a broader foundation, the free [AI Fundamentals course](https://flaviocopes.com/courses/ai-fundamentals/) explains models, tokens, context, agents, tools, and verification from the beginning.