Working playground

Validation & Leakage Playground

Change the dependency in the data and the split strategy independently. The lab flags designs that are computationally valid but scientifically unsafe.

Lab concept guide

What to observe while you experiment

Validation design should imitate the dependency structure of future use. Random splitting may be safe for independent rows but unsafe when rows share subjects/groups or temporal relationships; computational validity is not the same as scientific validity.

MechanismChange the data dependency and split strategy independently, then inspect exactly which related observations cross the train/validation boundary.

Failure modeUsing a standard random split by habit even when groups or time create information dependence.

VerificationAudit split indices for group/time overlap and compare scores from safe versus unsafe designs on the same data-generating process.

Experiment deliberately

Create grouped or temporal dependence, predict which split will leak information, then run both and explain any score inflation from the geometry of the split.

Data dependencyValidation methodFolds

Ready.