Why is “wrong” not enough information for learning?

examplepredictionlossgradientchanged parameters

01 · Smallest useful mechanism

A loss function turns the quality of one prediction into an optimization target.

Accuracy says only whether the winning class was correct. Learning needs a graded signal. For a target class with predicted probability p, cross-entropy uses −log(p). Raising the target probability lowers the loss smoothly, while pushing it close to zero creates a large penalty. Averaging this quantity across examples gives the optimizer one declared objective.

Cross-entropy is small when the model assigns high probability to the observed class and grows sharply when it is confidently wrong.

02 · Experiment

Test the prediction

Calculate by hand · cross-entropy curve

The probability slider produces an exact −log(p) value for one observed class and makes confidence-sensitive error visible.

Two-class teaching example

How much probability did the model assign?

Target probability p0.50

Target class p: 0.50
Other class 1−p: 0.50
Top-one verdict: Target class wins
−log(p): 0.693

In this binary example the classes cross at 0.5. With three or more classes, the target can rank first below 0.5; the loss still reads its exact probability.

Cross-entropy curve

p=.01loss 4.605

p=.50loss 0.693

p=.99loss 0.010

03 · Reason from evidence

Aggregate validation loss improves, but recall falls sharply for a rare class whose misses are expensive.

Your brief: Explain why the result is possible and specify what must change in the evaluation contract before anyone calls the model better.

Evidence your answer should use

Per-class counts, loss, precision, and recall
The reduction, weighting, masking, and sampling rules
Calibration or threshold behavior for the rare class
A cost-sensitive acceptance criterion agreed before retuning

There is no reveal button here. A strong response names the claim, the evidence that would test it, and what the evidence cannot establish.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

Your response

04 · Check your understanding

What happens to −log(p) as the target probability approaches zero?

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.on demand

Next: Once loss defines downhill, derivatives tell us which parameter movement is locally uphill.