How does a final loss reach every weight that helped produce it?

examplepredictionlossgradientchanged parameters

01 · Smallest useful mechanism

Backpropagation reuses local derivatives to assign credit through a computation graph.

A network is a composition of weighted sums and nonlinear functions. The chain rule says that sensitivity through a composition is the product of local sensitivities. Reverse-mode differentiation starts at one scalar loss, walks the recorded graph backward, and accumulates a gradient for every parameter that contributed to the result.

Values move forward; an upstream loss derivative moves backward and is multiplied by each operation’s local derivative.

02 · Experiment

Test the prediction

Exact computation graph · one ReLU hidden unit

The activity declares every weight and activation, then applies local derivatives and the chain rule to all parameters.

Gradient-flow preset

Computation graph · click a node

Forward pass

Data becomes a prediction

Parameters transform the input into intermediate activations. Nothing learns yet; this pass only computes values.

p = 0.7027

Parameter gradients

Analytic backpropagation and central finite differences should agree.

parameter	current	analytic gradient	numeric check	next value
w₁	0.5000	-0.4757	-0.4757	0.5951
b₁	0.2000	-0.2379	-0.2379	0.2476
w₂	0.8000	-0.3568	-0.3568	0.8714
b₂	-0.1000	-0.2973	-0.2973	-0.0405

Backpropagation is organized bookkeeping

Each operation needs only its local derivative and the gradient arriving from downstream. The chain rule composes those small facts into parameter gradients across an entire network.

Loading the learning experiment…

03 · Reason from evidence

The backward pass reports nonzero gradients, yet a snapshot shows that every weight is unchanged after the batch.

Your brief: Trace the training contract from loss to stored parameters. Name the checks that would locate the first missing state transition.

Evidence your answer should use

Whether the optimizer step ran and was skipped by any guard
Whether the intended parameters belong to the optimizer
Gradient, update, and parameter values for one named weight
Mixed-precision scale, clipping, and zeroing events around the step

There is no reveal button here. A strong response names the claim, the evidence that would test it, and what the evidence cannot establish.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

Your response

04 · Check your understanding

What does backpropagation produce directly?

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.on demand

Next: Efficient gradients let hidden layers learn features. Next we see how one new representation solves XOR.