Lesson 07 · The model makes a choice
Training target revealed
The animal did not cross the street because it was too tired
01 · Smallest useful mechanism
The transformer finishes with one contextual vector at each position. For generation, the last prompt position matters most: the language-model head compares its state with every token in the vocabulary and assigns each a raw score called a logit. Softmax turns relative scores into a distribution. The model has now finished this prediction step; a decoding rule decides what happens next. Greedy decoding always takes the highest-probability token. Sampling treats the distribution as a weighted lottery, often after controls such as temperature or top-p reshape the choices. This is why the same model and prompt can produce different continuations. These probabilities describe possible next tokens under the model—not truth, confidence in an entire answer, or guaranteed quality.
A final token state becomes alternatives, then a separate decoding rule selects one.
02 · Experiment
Calculate by hand · compare with local AI
Offline logits are curated teaching data. When enabled, the live lane queries Bonsai through Ollama as a separate comparison.
03 · Production-minded practice
Commit to a written claim and evidence plan. The Lab stores the draft locally and never sends it to a server.
Your learning artifact
Stored only in this browser. No account required; course reset does not delete it.
04 · Check your understanding
Next: After one token is appended, the same forward pass repeats—and reuse becomes a systems problem.
Static teaching dataset—not live model output
These illustrative probabilities stay available without Ollama and remain intentionally separate from live results.
Transformer blocks created the representation used here
After many attention and MLP blocks update the residual stream, an output projection creates vocabulary logits and softmax turns them into probabilities.
Prompt
The animal did not cross the street because it was too
Illustrative distribution — not live model output
Candidate distribution
A middle dot marks a leading space.
·tired
p = 48.00%
Training signal
Cross-entropy becomes large when the selected distribution assigns little probability to the correct next token.
loss = −ln(0.4800)
0.734
nats
