Trace a transformer prediction

Open Anatomy of a Transformer in the app.

Start with a short tokenized prompt such as [the, cat, sat]. At every position, a decoder-only transformer predicts the following token.

Each token ID becomes an embedding and receives position information. Masked self-attention lets a position combine earlier context without looking at future target tokens. A feed-forward network transforms each position. Residual paths carry information around both operations.

At the end, the model produces one score for every token in the vocabulary. Softmax turns those scores into probabilities.

Suppose three candidate scores become these probabilities:

" on"   0.60
" near" 0.25
" blue" 0.15

Training compares this distribution with the actual next token. Generation chooses one candidate using the configured sampling rule, appends it, and runs the prediction path again.

Write one sentence in your notebook for each stage: input IDs, embeddings, attention, blocks, output scores, next-token choice.

Change one prompt token and inspect the next-token distribution again. The difference shows context affecting computation. It does not prove that one attention head alone caused the final choice.

The model stores parameters, not completed answers. A generated paragraph is many repeated conditional predictions.

What to observe: the model produces a distribution of possible next tokens, not one stored answer.