What we are building

We are going to build a small language model on your Mac with Language Model Builder.

The goal is not to replace ChatGPT or Claude. The goal is to explain one complete learning system with evidence you collected yourself.

Our experiment follows this chain:

text → tokens → training examples → next-token loss
     → updated parameters → checkpoints → sampled text → evaluation

Later, supervised fine-tuning and preference training will shape how the same model responds. They do not replace tokenization, pre-training, or evaluation.

Open the app and choose Build a model. Do not start training yet.

The built-in textbook is useful when you want to pause the lab and revisit one idea without leaving the app.

Create a new project with a simple name such as Tiny Stories Lab. The name should help you identify this experiment later.

Write one hypothesis before changing a setting:

A small model trained on short stories will move from random tokens toward grammatical story continuations, but it will remain unreliable outside that domain.

By the end, you will have:

• the exact tokenizer, data split, and model blueprint
• training and validation curves
• samples from early, middle, and late checkpoints
• SFT and preference comparisons
• a report separating observations from explanations

The report matters as much as the weights. A model file cannot tell us which data it saw, which controls stayed fixed, or why we trust a conclusion.