Calibration, Predictive Uncertainty & Model Interpretation – Practice Quiz

Calibration, predictive uncertainty and model interpretation: evaluate probability forecasts using Brier score and log loss, diagnose reliability diagrams, choose cost-sensitive thresholds, interpret partial dependence cautiously and construct split conformal prediction intervals under exchangeability. Explore rare-class evaluation, data leakage and the limits of marginal coverage. Includes original single-select and multiselect scenarios, worked reasoning and explanations of each distractor. Prerequisites: classification, validation, conditional probability and neural-network generalization. Authored from the saved Statistics & Probability syllabus; intended for progressive independent practice.

Further reading: Probability calibration, Model evaluation, Angelopoulos and Bates: Conformal prediction, Partial dependence and ICE.