WEBVTT

00:00.000 --> 00:16.240
In session 6 we did a forward pass, we fed numbers in.

00:16.240 --> 00:21.040
The network produced a prediction, 7.5, now comes a question.

00:21.040 --> 00:25.240
What if 7.5 was wrong?

00:25.240 --> 00:31.400
Suppose the correct answer was 5.0, the network predicted 7.5, the loss, our measure of

00:31.400 --> 00:38.720
wrongness, is 6.25, the network was wrong, now we need to fix it.

00:38.720 --> 00:42.400
The network has many weights, which ones caused this error?

00:42.400 --> 00:46.800
We can't just guess, we need a systematic way to trace the error back to the weights that

00:46.800 --> 00:48.320
caused it.

00:48.320 --> 00:52.880
That algorithm is called back propagation.

00:52.880 --> 00:58.080
Back of the weights is a team of chefs, they all contributed to the final dish, the prediction,

00:58.080 --> 01:00.880
the dish came out wrong, you don't fire everyone.

01:00.880 --> 01:06.040
You need to figure out who added too much salt.

01:06.040 --> 01:10.640
Back propagation traces the error backward through the network, from the output, back through

01:10.640 --> 01:17.440
the hidden layer, all the way to each weight, this is the backward pass.

01:17.440 --> 01:22.160
Back propagation measures how much each weight contributed to the error, a weight that pushed

01:22.160 --> 01:26.360
the prediction far in the wrong direction gets more of the blame, a weight with little

01:26.360 --> 01:31.080
influence gets less.

01:31.080 --> 01:35.800
Each weight gets a small nudge, a tiny adjustment in the direction that would have made the prediction

01:35.800 --> 01:41.040
less wrong, not a big change, a nudge.

01:41.040 --> 01:45.520
Now we run another forward pass with the adjusted weights, the prediction is a little closer

01:45.520 --> 01:51.920
to the target, still wrong, but less wrong, we do it again, and again.

01:51.920 --> 01:57.240
Each time, the weights improve by a tiny amount.

01:57.240 --> 02:03.160
A network with millions of weights learns by doing this millions of times, forwardpass.os,

02:03.160 --> 02:09.800
make a prediction, backward pass, learn from the mistake, forward, backward, forward, backward.

02:09.800 --> 02:10.400
That is training.