Learning Curves, Bias & Variance
Learning curves and complexity curves help diagnose underfitting, overfitting and whether additional data are likely to help. The topic is split into focused lessons so definitions, implementation decisions, diagnostics and common failure modes can be learned separately.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
High biasTraining and validation performance are both poor and close together; increasing model capacity or improving features may help. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
02High varianceTraining performance is strong but validation performance is substantially worse; regularisation, simpler models or more representative data may help. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
03Learning curvesPlot performance versus training-set size. A persistent validation gap suggests variance; both curves plateauing poorly suggests bias. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
04Complexity curvesVary depth, regularisation strength, feature count or another capacity parameter and compare training versus validation performance. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
05Data qualityCurve shape can also reflect label noise, dataset shift or leakage, so diagnostics must be interpreted in context. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.