Explainability & Model Interpretation
Explainability techniques describe global model behaviour or why a specific prediction changed, but explanations are approximations whose assumptions must be understood. The topic is split into focused lessons so definitions, implementation decisions, diagnostics and common failure modes can be learned separately.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
Global importancePermutation importance measures performance loss when a feature is disrupted; tree impurity importance is fast but can be biased. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
02PDP and ICEPartial Dependence averages predictions across a feature grid; ICE shows individual trajectories and exposes heterogeneous effects. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
03SHAP conceptsShapley-based attributions distribute prediction difference among features using a game-theoretic framework; background/reference choices matter. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
04Local surrogate methodsLIME approximates the model locally around one instance; fidelity and perturbation strategy should be checked. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
05Deep visual explanationsIntegrated Gradients, Grad-CAM and attention visualisations expose gradients/activation patterns, but saliency is not identical to causal relevance. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.