Model Building & Algorithm Learning

Gradient Descent Simulator

Optimise a linear regression model on a visible loss surface and compare batch, stochastic and mini-batch gradient updates.

7 · Regression 8 · Clustering 9 · Decision boundaries 10 · Neural networks 11 · Gradient descent 12 · Ensembles 13 · Step-by-step

Lab concept guide

What to observe while you experiment

Gradient descent updates parameters in the direction that locally reduces a loss. Learning rate controls step size; batch, stochastic and mini-batch variants differ in how much data estimates each gradient, changing noise and convergence behaviour.

MechanismAt each update, connect the current parameters to the loss, gradient direction and resulting parameter step on the visible surface.

Failure modeIncreasing the learning rate until loss diverges or mistaking noisy stochastic updates for incorrect optimisation.

VerificationFor a simple linear-regression point, verify the sign of one gradient/update and confirm that a sufficiently small step moves toward lower loss.

Experiment deliberately

Start with a small learning rate, predict the next parameter movement, then raise the rate until overshoot appears. Compare batch and stochastic paths.

Update methodLearning rate0.035Mini-batch size10Noise8Seed

Try the learning-rate extremes. A tiny rate moves slowly; a useful rate converges; an excessive rate can overshoot or diverge.

Ready at the initial parameters.

Loss surface

Each point on the surface is the MSE for one intercept/slope pair. The path shows optimisation history.

Gradient Descent Simulator

What to observe while you experiment

Loss surface

Current fitted line

Loss by update