← All work

How a model learns

A real training run drives the film: the model's error falls from 0.680 to 0.0059 in 60 steps.

Plays with sound where your browser allows; the switch at the top turns it off.

The question

How does a machine-learning model learn? I wanted to show it in a film that works with the sound off.

A tiny model fits a line through 12 made-up dots, and a ball rolling into a valley shows each correction.

  1. The first guess0.680loss

    The line starts at a slope of −0.60, tilted the wrong way. Its error (its loss) is 0.680.

  2. The swing, step 71.42slope

    With momentum, the line swings past the best slope to 1.42, then back, like a weight on a spring.

  3. After 60 steps0.0059loss

    The lowest possible for a straight line through these dots. Never zero: the dots don't sit on one line.

The numbers drive the film

Facts first, from Google's Machine Learning Crash Course. A real training run drives the film, so nothing moves that the numbers don't. It took 17 versions.

  • Google, Machine Learning Crash Course: loss and gradient descent
  • Gabriel Goh, "Why Momentum Really Works", Distill, 2017
  • A small training script whose output drives every frame
  • Designed and animated in code (Remotion)

One code base, two palettes

The film shipped on 4 November. A day later it was recoloured for the blog from the same code: one render setting switches the palette. Timing, copy, numbers and sound are unchanged.

Step 6 of the film in the original palette
The film, 4 Nov
Step 6 of the film in the blog palette
The blog version, 5 Nov

It became a post and a tool

Explain it back: machine learning/skills/explain-it-back ↗A free Claude skill, published on GitHub.
Cover of the article 'How AI learns: by being wrong, a little less each time'
The article: "How AI learns: by being wrong, a little less each time"

Self-initiated work. Sources: Google, Machine Learning Crash Course; Gabriel Goh, "Why Momentum Really Works", Distill, 2017. The dots are made up for the film; every number comes from the training run.