Regularisation & Early Stopping
Regularisation controls effective model complexity so a learner captures reproducible structure instead of memorising noise. The topic is split into focused lessons so definitions, implementation decisions, diagnostics and common failure modes can be learned separately.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
L2 regularisationPenalises squared parameter magnitude, encouraging smaller distributed weights and smoother solutions. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
02L1 regularisationPenalises absolute parameter magnitude and can drive coefficients exactly to zero, creating sparse models. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
03Elastic NetCombines L1 and L2 to balance sparsity with stability among correlated features. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
04Dropout and augmentationDeep-learning regularisers inject stochastic perturbations or varied examples, discouraging reliance on fragile pathways. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
05Early stoppingMonitors validation performance and stops optimisation when additional training no longer generalises. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.