Labs / Computer Science
Gradient Descent
A straight line is trying to fit a cloud of points. On the left it nudges its slope and intercept to shrink the total squared error; on the right, those same two numbers are a marble rolling downhill on the loss surface. Add your own points, change the learning rate, and step or run the descent.
m —b —MSE —descending
What to try
- Crank the learning rate all the way up. Why does the marble shoot off the surface instead of settling into the valley?
- Set a tiny learning rate and press Run. Is the line still improving, or has it basically stalled?
- Click to drop a single point far from the others. How much does one outlier drag the best-fit line toward it?
- Where on the loss surface does the marble slow down, and what does that tell you about the slope of the hill there?
- Switch from Rising to Scattered data. What happens to the shape of the valley when the points have no real trend?