Start a model lab notebook

Create a Markdown file named model-lab.md, or use any notes app you trust. This will become the experiment record.

Start with these headings:

# Tiny Stories Lab

## Goal
## Mac and app version
## Run manifest
## Model configuration
## Tokenizer and dataset
## Data split
## Training log
## Checkpoint samples
## Evaluation settings
## Fine-tuning
## Limitations

Write the goal in one sentence. Example: “Observe how a small transformer changes during pre-training, then test whether SFT improves instruction-shaped responses.”

Record your Mac chip, memory, macOS version, and app version. These details help explain timing differences when you compare runs weeks apart.

Paste the three baseline prompts from the previous lesson.

Create a run manifest before training:

Run ID: starter-stories-v1
Random seed:
Tokenizer and vocabulary size:
Training, validation, and test data IDs:
Context length:
Model shape:
Batch size:
Step budget:
Learning-rate settings:
Sampling settings:

Record units. “Memory: 18” and “speed: 900” are useless without MB, GB, tokens per second, or another defined unit.

Keep observations separate from explanations:

Observation: validation loss rose after step 12,000.
Possible explanation: the model began fitting training-specific patterns.
Next test: compare the 10,000 and 12,000 checkpoints under fixed sampling.

Never edit an old result to match a later interpretation. Add a dated correction instead.

My habit is to timestamp every notebook entry. When a checkpoint sample looks wrong, I can see whether I changed a sampling setting the same hour.

If you forget the run manifest and start training anyway, you will regret it at the first checkpoint comparison. Fill the manifest now even if some fields say TBD. Empty fields are honest. Missing fields are guesswork later.

Add one line under Training log before the run starts: No training yet. Notebook created on <date>. That tiny entry proves the file existed before results tempted you to rewrite history.

When a friend asks what you learned, this notebook is the answer. Without it, you remember the best sample and forget the configuration that produced it.

What to observe: the notebook begins with facts and questions, not a conclusion written in advance.