Free course

Build a Language Model on Your Mac

Use Language Model Builder to design, train, inspect, fine-tune, and evaluate a small language model entirely on your Mac.

6 modules · 29 lessons · No signup

Download this course as PDF or EPUB

Read it offline, on your Kindle, or print it. Join my free newsletter and you get this course plus every other book and course I published.

Download PDFDownload EPUB
~~~

Your progress

of lessons completed.

Course completed.

What you'll learn

Complete one documented language-model experiment from blueprint and data selection through pre-training, fine-tuning, comparison, and an honest lab report.

Course contents

  1. Plan the experiment

    Install the app, set realistic expectations, and define what you will measure.

    1. ✓ What we are building
    2. ✓ Install the app and check your Mac
    3. ✓ What a small model can and cannot do
    4. ✓ Start a model lab notebook
    5. ✓ Check your understanding: plan the experiment

  2. See what the model sees

    Inspect tokens, embeddings, attention, and the transformer path.

    1. ✓ Tokens and token IDs
    2. ✓ Run the tokenizer playground
    3. ✓ Explore embedding space
    4. ✓ Trace a transformer prediction
    5. ✓ Check your understanding: model foundations

  3. Design the run

    Choose the model shape, tokenizer, data, and practical resource budget.

    1. ✓ Choose a model blueprint
    2. ✓ Parameters, memory, and training estimates
    3. ✓ Choose a tokenizer
    4. ✓ Select and prepare a dataset
    5. ✓ Check your understanding: design the run

  4. Pre-train and observe

    Run training over time, read loss, sample checkpoints, and diagnose results.

    1. ✓ Start a pre-training run
    2. ✓ Pause and return later
    3. ✓ Read training and validation loss
    4. ✓ Checkpoints and sampling
    5. ✓ Diagnose poor results
    6. ✓ Check your understanding: pre-train and observe

  5. Shape model behavior

    Use SFT, preference data, chat, and inspection tools deliberately.

    1. ✓ Run supervised fine-tuning
    2. ✓ Preference data and DPO
    3. ✓ Chat with your model
    4. ✓ Inspect the model with visualization tools
    5. ✓ Check your understanding: shape behavior

  6. Compare and report

    Compare checkpoints fairly and document what the experiment proved.

    1. ✓ Compare checkpoints fairly
    2. ✓ Write the model lab report
    3. ✓ Final check: compare and report