Lesson 09 · The error travels backward
Training target revealed
The animal did not cross the street because it was too tired
01 · Smallest useful mechanism
During training, ordinary text supplies many small practice questions. At each position, the model sees preceding tokens and predicts what came next. If the observed token receives low probability, the loss is large; if it already receives high probability, the loss is small. Backpropagation follows the recorded computation backward and calculates how each participating parameter contributed to the loss. An optimizer then makes a very small update. This repeats across enormous numbers of token examples. The model learns distributed statistical patterns rather than saving every sentence as a database record. Loss, backpropagation, and optimization are different: loss measures error, backpropagation computes gradients, and the optimizer decides how to use them. During ordinary inference, no update occurs.
Loss measures the miss, backpropagation assigns responsibility, and the optimizer changes weights.
02 · Experiment
Calculate by hand · exact loss step
The probability, cross-entropy, gradient direction, and update are a small transparent training example rather than a captured model update.
Observed target · “tired”
p(tired)
0.18
loss = −ln(p)
1.715
The learning chain
Loss measures the miss. Backpropagation computes which parameters contributed to it. The optimizer turns those gradients into a bounded update.
03 · Production-minded practice
Commit to a written claim and evidence plan. The Lab stores the draft locally and never sends it to a server.
Your learning artifact
Stored only in this browser. No account required; course reset does not delete it.
04 · Check your understanding
Next: The capstone asks you to explain which representation changes at every stage. The How Machines Learn course provides the deeper perceptron-to-backpropagation path.
