Training, Validation & Model Selection

Experiment History

Keep a lightweight local record of pipeline, model, validation and tuning experiments and compare what changed.

14 · Pipeline 15 · Compare 16 · Tune 17 · Cross-validation 18 · Bias/variance 19 · History

Lab concept guide

What to observe while you experiment

Experiment history turns iterative modelling into an auditable comparison: each run should record data/configuration, validation design, model settings, metrics and the change from the previous run. Without that lineage, improvement claims are hard to reproduce.

MechanismSave runs with enough configuration and result metadata to reconstruct what changed and compare like with like.

Failure modeComparing scores from runs that used different splits/metrics/data versions without recording those differences.

VerificationSelect two saved runs and verify that the displayed delta can be explained from their recorded configuration and identical evaluation protocol.

Experiment deliberately

Save a baseline, change exactly one setting, save again and write a one-sentence causal hypothesis for the observed metric difference.

Local experiment notebook. Training, Validation & Model Selection workbenches can save runs here. History stays in this browser via localStorage and can be exported as CSV or JSON.

Saved experiments

Select up to three runs to compare.

Selected-run comparison

Train, validation and test metrics when available.