Data Understanding & Feature Selection

Correlation Analysis & Feature Selection

Measure linear or monotonic association, inspect a correlation matrix, and select useful non-redundant features using train-side correlation evidence.

Dataset Feature engineering Correlation selection Dimensionality reduction Explainability

Lab concept guide

What to observe while you experiment

Correlation screening combines two distinct checks: feature–target association and feature–feature redundancy. Pearson captures linear association; Spearman captures monotonic rank association. Neither establishes causality or guarantees predictive usefulness.

MechanismInspect the correlation matrix and raw scatter plots, rank features by target association, then reject a candidate when it is too redundant with a stronger already-selected feature.

Failure modeSelecting features on the final test set, using correlation without looking at relationship shape, or discarding nonlinear/interaction features solely because univariate correlation is weak.

VerificationFor a few paired values, compute or reason through the correlation/ranks manually; then confirm that changing the minimum association or maximum redundancy threshold changes only train-side selection.

Experiment deliberately

Add the influential outlier, compare Pearson with Spearman, then run the feature selector step by step and explain why each candidate is kept or rejected.

Correlation measures association, not causation. Pearson correlation measures linear association; Spearman correlation measures monotonic association based on ranks. A feature can be important despite weak correlation when its relationship is nonlinear or interaction-dependent.

MethodRowsNoise0.42Seed Add influential outlier

Correlation matrix

Inspect feature–feature redundancy as well as feature–target association.

Scatter + fitted line

Select a feature to inspect the raw relationship behind the coefficient.

Feature

Correlation-based feature selection

First require enough feature–target association, then reject features that are too correlated with an already selected stronger feature.

Minimum |feature–target correlation|0.25Maximum |feature–feature correlation|0.85 Step-by-step selection

Ready.

PearsonUse for linear association between numeric variables; sensitive to influential outliers.

SpearmanUses ranks, so it captures monotonic relationships and is less driven by exact numeric spacing.

Feature selectionFit thresholds using training data only. Never use the final test set to choose features.

LimitationCorrelation screening is univariate: nonlinear effects, interactions and conditional usefulness can be missed.