Awesome Testing

Lesson 06 · Updates accumulate

Where do attention and MLP updates go after each sublayer?

Unfinished prompt

The animal did not cross the street because it was too

01 · Smallest useful mechanism

A shared page gathers useful notes

Imagine every token position carrying a working page through the transformer. The page begins with that token’s initial vector. Each attention layer can add a note gathered from other positions, and each MLP can add a locally computed note. The page is the shared channel through which components communicate; it is not readable prose. Its size stays the same while its contents become increasingly contextual. For next-token prediction, the final page at the last prompt position goes to the language-model head. We can project earlier pages into vocabulary space to watch candidates rise and fall, but that projection is an interpretability probe. It helps form hypotheses about how a prediction develops; it is not a recording of private thoughts.

Every block adds updates to temporary token states that keep the same shape; those updates are not automatically evidence in the factual sense.

02 · Experiment

Test the prediction

Calculate by hand · exact residual updates

The walkthrough is illustrative and internally consistent. Model-backed hidden-state inspection remains future research.

Loading the experiment…

03 · Production-minded practice

Interpret a hidden-state probe carefully

Commit to a written claim and evidence plan. The Lab stores the draft locally and never sends it to a server.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

04 · Check your understanding

What persists between transformer blocks?

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.

Next: The final state at “too” is ready to be compared with every vocabulary token.