Calibration, Thresholds & Decision Costs
Calibration, Thresholds & Decision Costs groups the core ideas a learner needs at the 9 · evaluation, metrics & diagnostics stage. Work through the lessons in order when new to the area, or use them independently as a reference when implementing an analysis.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
Probability calibrationProbability calibration is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
02Reliability diagramsReliability diagrams is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
03Platt scalingPlatt scaling is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
04Isotonic calibrationIsotonic calibration is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
05Temperature scalingTemperature scaling is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
06Threshold optimisationThreshold optimisation is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
07Cost-sensitive thresholdsCost-sensitive thresholds is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
08Abstention / reject optionAbstention / reject option is a practical concept within Calibration, Thresholds & Decision Costs. It helps turn the broader workflow stage “9 · Evaluation, Metrics & Diagnostics” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.