Open-weight AI models: what open weights means and how they work

Meta released a new AI model called Muse Glimmer in August 2026.

It is a 30-billion-parameter model designed for local AI agents. It can read text and images, call tools, and recover when a tool fails.

It’s also an open-weight model.

The full model would normally need more than 55 GB of memory. Meta also released quantized versions that fit in less than 20 GB.

This means you can download one of those versions and run it on your own computer. No request has to travel to Meta’s servers.

You’ll see the phrase “open weights” in almost every model announcement now. It sounds like open source, and people often use the two to mean the same thing. They’re related, but they describe different things.

To understand the difference, we first need to understand what weights are.

What are model weights?

Think of a large language model as a huge audio mixing desk.

The desk has billions of small knobs. Each knob changes the signal a little as it passes through.

Before training, those knobs contain mostly random values. The model is not useful yet.

During training, the model sees text, images, code, or other data. It tries to predict an answer, measures how wrong it was, and adjusts the knobs.

It repeats this process many times.

Those learned knob values are the weights.

A real model does not store knobs, of course. It stores billions of numbers inside tensors, which are multi-dimensional arrays of numbers.

When you see a model described as 8B or 30B, the B means billion parameters. Most of those parameters are weights. Muse Glimmer has about 30 billion of them. A frontier model can have hundreds of billions, or trillions.

Each value means almost nothing alone. When you ask a model to explain a JavaScript function, there’s no single weight holding “JavaScript knowledge”. Your prompt moves through many layers, and the learned values shape the result at every step.

The weights are not the code that trains the model. They are the expensive result of training, the model’s learned state. The architecture may be described in a paper and the inference code may be on GitHub, but without the trained weights you have an untrained machine.

If you want to try training a small model yourself on a Mac, see Build a language model on your Mac with Language Model Builder.

The architecture and the weights are different things

The architecture describes how the model is built.

It defines the layers, connections, attention mechanism, and the path data follows through the model.

The weights are the learned numbers placed inside that architecture.

Two models can use a similar architecture but have very different weights. They will behave differently because they learned from different data or were trained for different goals.

The runtime is another separate piece. It is the software that loads the architecture and weights, then performs the calculations needed to generate an answer.

So when you run a local model, you need several parts:

• the model architecture and configuration
• the weights
• a tokenizer that turns text into tokens
• generation settings and instructions for loading the model
• an inference runtime such as Ollama, llama.cpp, MLX, or Transformers
• enough memory and compute to run everything

An API hides all of this behind an HTTP request.

An open-weight release lets you bring those pieces onto your machine.

What does open-weight mean?

An open-weight model is a model whose learned weights are available to download. What you’re allowed to do with them depends on the license, which I cover further down.

Muse Glimmer is a useful example because Meta released its artifacts under the permissive Apache 2.0 license. Its model card includes full-precision weights, two 4-bit versions, the vision encoder, and a small companion model used to speed up generation.

Open access, open weights, and open source

AI releases sit on a spectrum.

A model can be free in a chat interface, or have a generous API, and still be completely closed. It can also publish its weights and keep its training data secret. So “open model” is too vague. You need to ask which part is open.

With normal software, source code is the preferred form for studying and changing a program. A compiled application is useful, but it does not tell you everything about how the program was created.

Model weights have a similar problem. You can run and modify them, but they don’t tell you which data produced them, how that data was filtered, or which training process was used.

The Open Source AI Definition from the Open Source Initiative asks for more than the final parameters. It includes the code used to train and run the system, along with enough information about the training data to let people understand and substantially recreate it. The OSI also has a page on the difference between open weights and open-source AI.

Open weights tell you that you can get the learned parameters. Open source AI should also give you the materials and freedoms needed to study, modify, and share the wider system.

The boundary is still debated. But “open weight” is the more precise term when a company releases the trained parameters without everything used to create them.

Why the training recipe matters

Suppose a lab releases a powerful coding model.

You can download it, run evaluations, fine-tune it on your code, and inspect its behavior across different prompts.

But several questions may remain unanswered:

• Which repositories were included in training?
• Were licenses respected?
• How was private information filtered?
• Which benchmark data appeared in the training set?
• Which synthetic data was generated by other models?
• How did the lab remove low-quality examples?
• Which reinforcement learning process changed the model after pretraining?

The weights contain the effects of the training process, not a readable history of it. That gap affects research, copyright, security, bias, and reproducibility.

You can study an open-weight model from the outside and inspect some of its internal behavior. You cannot look at a trillion numbers and recover the original training dataset.

What happens when you download a local model?

The model repository normally contains one or more very large files.

Files in the safetensors format hold tensors and nothing executable. This makes the format safer and faster to load than older Python pickle-based model files.

For local inference, you will also see GGUF files.

GGUF stores the model tensors together with metadata in one file. Runtimes including llama.cpp, Ollama, and LM Studio can load it. If you want to use that LM Studio server from a coding agent, OpenCode can talk to it.

The file may have a name like this:

model-Q4_K_M.gguf

The Q4 part tells you the weights were quantized to around 4 bits. The rest identifies the quantization method.

The runtime reads the configuration, allocates memory, loads the weights, and waits for a prompt.

When you enter text, the tokenizer converts it into token IDs. The model passes those tokens through its layers and uses the weights to calculate probabilities for the next token.

It selects a token, adds it to the sequence, and repeats.

Nothing has to call a cloud API. The calculations happen on your hardware.

If you want to try that process, the free Local AI Models Course walks through installing Ollama, downloading a model, and talking to it through the local API.

Why quantization matters

Weights take a lot of memory.

A 30-billion-parameter model stored with 16 bits per weight needs about 60 GB just for the raw weight values:

30 billion × 16 bits ÷ 8 = 60 GB

The exact file and memory size varies with the model and format. But this estimate tells us why full-precision models are difficult to run at home.

Quantization stores the weights using fewer bits.

At roughly 4 bits per weight, the same estimate becomes:

30 billion × 4 bits ÷ 8 = 15 GB

Real files also need metadata, scales, and sometimes other model components. Meta’s compact Muse Glimmer release is around 17 GB, with the full local setup targeting computers with 24 GB or 32 GB of memory.

Quantization is a tradeoff. A smaller file uses less memory and may run faster, but reducing precision can change model quality.

Good quantization methods preserve much more quality than the simple bit count suggests. Still, you should test the exact model and quantization on your task.

Community quantizations are one reason open-weight models spread quickly. A model released for datacenter GPUs often appears within days in formats for Apple silicon, consumer GPUs, or CPU inference.

I explained the complete memory calculation in how much VRAM you need to run an LLM locally.

What open weights let you do

The obvious benefit is local execution. But open weights change more than where a prompt runs.

Keep the model version you tested

A cloud provider can update a model, change its price, or stop serving it.

With downloadable weights, you can keep the exact version you tested. Your application does not change because a provider silently replaced the model behind an API name.

This is useful for repeatable tests, research, and products that need stable behavior.

Keep private data on your machine

A local model can process notes, source code, customer records, or documents without sending the prompt to a model provider.

But local does not automatically mean private.

An agent can still call a search API, upload a file, write logs, or send tool results elsewhere. You must inspect the complete data flow, not only the model.

Choose who runs it

The same model may run through several runtimes and hosting providers. You can compare price, speed, location, privacy terms, and reliability, and move between them without changing the model.

You can start locally, move it to your own server, or use a hosted inference service when traffic grows.

GLM-5.3 is a good example. You can use it in Z.ai’s ZCode app, through Z.ai’s API in Claude Code, or on your own GPU server with vLLM.

Fine-tune it

You can continue training an open-weight model for a specific task. You might adapt it to your company’s terminology, an internal framework, a writing style, support ticket classification, or a language the original model handles poorly.

Fine-tuning every weight is expensive. Techniques such as LoRA train a much smaller set of additional parameters, which makes customization practical without repeating the original training run.

Study and test it

Researchers can run controlled experiments without sending prompts through a private API. They can inspect activations, test modified inference methods, remove or replace layers, compare quantizations, and search for failure patterns.

A community can also make one release work across Apple Silicon, NVIDIA GPUs, CPUs, phones, and embedded devices, and find problems the original creator missed.

Use small models as building blocks

Not every task needs the strongest cloud model.

A smaller open-weight model can classify messages, extract fields, summarize private documents, or handle one step inside a larger system.

This is where local models are most interesting to me. They only need to perform one useful task reliably enough, not win every benchmark.

The license is as important as the download

Every model comes with a license or usage terms. A downloadable model can still restrict what you may do with its weights.

Check whether the license permits:

• commercial use
• redistribution of the original or modified weights
• hosted services
• fine-tuning, quantizing, or converting to another format
• use above a certain company size or number of users
• use in particular industries or applications

Some model families use familiar licenses such as Apache 2.0. Others use custom licenses with additional restrictions.

The Hugging Face license documentation lists standard software licenses alongside model-specific agreements. That alone shows why “available on Hugging Face” does not tell you the legal terms.

Treat the model card and license as part of the dependency. You would not add a software package to a commercial application without checking its license. Do not skip that check because this dependency contains billions of numbers.

Recent large releases

Qwen3.8

On July 19, 2026, the Qwen team announced Qwen3.8, a model with up to 2.4 trillion parameters, and said it would go open-weight soon. The weights landed in August.

Qwen/Qwen3.8-2.4T-A95B went up on August 12. It is a 2.4-trillion-parameter mixture-of-experts model with 95 billion active parameters. The license is a custom Qwen3.8-Max license, not Apache.

Qwen/Qwen3.8-27B followed on August 14. That one is a dense multimodal model under Apache 2.0. Read both cards before you build around either download.

The earlier Qwen3 release included dense models from 0.6 billion to 32 billion parameters, plus mixture-of-experts models reaching 235 billion total parameters. Qwen released those under Apache 2.0 and documented ways to run them with llama.cpp, Ollama, LM Studio, vLLM, and SGLang.

The 2.4-trillion-parameter model moves research forward and gives inference providers another frontier model. The 27-billion-parameter one can run on hardware you own, which makes it the more useful of the two for most individual developers.

Kimi K3

Moonshot AI announced Kimi K3 on July 16, 2026. It has 2.8 trillion total parameters, native vision, and a 1-million-token context window.

Moonshot released the full weights on July 27, 2026 under the Kimi K3 License. You can download them from moonshotai/Kimi-K3 on Hugging Face. The repo is ungated, and the files are about 1.56 TB of MXFP4 safetensors split across 96 shards.

Kimi K3 uses a mixture-of-experts architecture with 896 experts, and activates 16 for each token. A routing system picks that smaller part for the current computation, so inference is cheaper than running all 2.8 trillion parameters every time.

The model is still huge. Moonshot documents vLLM and SGLang support and recommends deployments with 64 or more accelerators, so most people will not run it on a laptop beside Spotify.

Open describes access and permissions. Accessible describes whether you can realistically use the model with the resources you have.

A model can be open-weight and impractical for a solo developer to host, while a smaller model with a custom license can be easy to run but less open.

Mistral Large 4

Mistral launched Mistral Large 4 in public API preview in October 2026, and published the weights at the end of the month on Hugging Face. It’s a mixture-of-experts model with about 1 trillion parameters and 49 billion active per token, under Mistral’s own license based on MIT.

It’s a good example of why “will be open-weight” and “is open-weight” are separate events. For most of October you could only use it through the API.

Why release weights for a model this large?

If almost nobody can run Kimi K3 at home, why release the weights?

Because running it on a laptop is only one benefit.

Inference companies can host it. Researchers can study it. Hardware vendors can optimize for it. Framework teams can add support. Quantization experts can reduce its memory needs. Other labs can build fine-tunes or distill its behavior into smaller models.

The weights also give the ecosystem a shared artifact. Improvements to serving, caching, kernels, and quantization do not need permission from the original lab. Moonshot says Kimi K3 introduces new attention and routing techniques, and that it is contributing support to the vLLM community.

When one lab releases a strong model, other model companies have to compete with something people can download.

What open weights do not give you

Open weights do not guarantee:

• strong output quality
• accurate answers
• unbiased training data
• low hardware costs
• fast generation on your computer
• permission to use the model however you want
• enough information to reproduce the training run

Open weights does not mean cheap

Model files need storage, and inference needs memory. Long context windows use even more. High traffic needs more GPUs, fast responses depend on optimized kernels and batching, and someone has to operate the system.

For a small model, local inference may reduce your costs. For a giant model, a hosted API may still be far cheaper than self-hosting. The advantage is that you can choose among providers, or bring the model in-house when your scale justifies it.

So don’t compare only the price per token. Count the hardware, power, engineering time, observability, scaling, failover, security and model updates too.

Open weights does not mean safe

Running a model yourself makes you the operator. You choose the runtime, secure the machine, install updates, measure quality, and decide what the model can access.

Self-hosting does not remove hallucinations. An agent can still delete a file, expose a secret, or follow a malicious instruction. The risk usually comes from combining the model with tools, broad permissions and weak review, whether the model runs on your laptop or someone else’s server.

For an agent that can change files or call external services, running locally does not remove the need for permissions and confirmation.

How I would use an open-weight model

I would not replace every cloud model with a local one.

I would start with small, frequent tasks where privacy and predictable costs matter more than frontier reasoning.

For example, I recently added an optional local LLM to a Swift app. The model turns factual activity records into one short daily sentence. If Ollama is missing or the model fails, the original data still works.

I like this boundary because the model adds value without making the product depend on it.

For a model like Muse Glimmer, I would also experiment with a private local agent. It could inspect files, interpret screenshots, and call a small set of approved tools.

I would keep destructive actions behind confirmation. I would also compare its results against a strong cloud model before trusting it with long coding tasks.

I would not use a local model when my laptop cannot run it comfortably, when a small number of API calls costs less than the hardware, or when the task needs the best available reasoning.

How to evaluate an open-weight model

When you see a new “open” model announcement, or find a model on Hugging Face, check these things before downloading it:

1. Can I download the weights today? A future release is still a promise.
2. License. Can you use, modify and redistribute it for your project?
3. Parameter count. Is the model large enough for the task and small enough for your machine? Total and active parameters are not the same thing.
4. Quantization. Which versions are available, how much memory do they need, and how much quality do they lose?
5. Runtime support. Are the tokenizer, configuration and inference code public, and does it work with Ollama, llama.cpp, MLX, Transformers, or your chosen server?
6. Base or instruction model. A base model may need more training before it behaves like a chat assistant.
7. Context length. Can it hold the input your task needs, and how much extra memory will that context use?
8. Model card. What was it designed for, how was it trained and evaluated, and what limitations does the creator list? The Hugging Face model card guide shows what a serious release should document.
9. Benchmarks you can reproduce. A score without the same harness, prompts, and tools is hard to compare.
10. Your own test set. Does it work on real examples from your application?

Do not choose a model from a leaderboard alone. The best model is the smallest one that performs your task well enough on hardware you can operate.

If you are deciding between local hardware and an API, I also built a local LLM vs API cost calculator and wrote out the complete cost comparison.

If you want to put this into practice, my free Local AI Models Course takes you from open weights to a working local AI feature with Ollama and Node.js.

For a broader foundation, the free AI Fundamentals course explains models, tokens, context, agents, tools, and verification from the beginning.

Want me to talk about your product? You can sponsor this site.