Missing Data: Concepts & Diagnosis
Missing Data: Concepts & Diagnosis groups the core ideas a learner needs at the 4 · data cleaning & missing data stage. Work through the lessons in order when new to the area, or use them independently as a reference when implementing an analysis.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
Missing-value representationsMissing values are not a single universal token. In pandas and NumPy they can appear as `NaN`, `pd.NA`, `NaT` or domain-specific sentinel codes such as -999. The first job is to convert all legitimate missing representations into an explicit, typed missing value without accidentally treating valid values as absent.
02Missingness rate by row and columnMissingness rate by row and column is a practical concept within Missing Data: Concepts & Diagnosis. It helps turn the broader workflow stage “4 · Data Cleaning & Missing Data” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
03Missingness patterns and matricesMissingness patterns and matrices is a practical concept within Missing Data: Concepts & Diagnosis. It helps turn the broader workflow stage “4 · Data Cleaning & Missing Data” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
04MCAR: missing completely at randomMCAR means the probability that a value is missing does not depend on observed or unobserved values. Under true MCAR, complete-case analysis can remain unbiased for many estimands, although it wastes information and reduces precision.
05MAR: missing at randomMAR means missingness may depend on variables you observed, but after conditioning on those observed variables it does not additionally depend on the missing value itself. Many likelihood and multiple-imputation methods rely on an MAR-style assumption.
06MNAR: missing not at randomMNAR means the probability of missingness still depends on the unobserved value even after conditioning on observed data. Standard imputation under MAR may then be biased unless the missingness mechanism is modelled or sensitivity analysis is performed.
07Why missingness mechanism mattersWhy missingness mechanism matters is a practical concept within Missing Data: Concepts & Diagnosis. It helps turn the broader workflow stage “4 · Data Cleaning & Missing Data” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.