mc_calibration#

bayesflow.diagnostics.mc_calibration(pred_models: Mapping[str, ndarray] | ndarray, true_models: Mapping[str, ndarray] | ndarray, model_names: Sequence[str] = None, num_bins: int = 10, label_fontsize: int = 16, title_fontsize: int = 18, metric_fontsize: int = 14, tick_fontsize: int = 12, epsilon: float = 0.02, figsize: Sequence[int] = None, color: str = '#132a70', num_col: int = None, num_row: int = None) → Figure[source]#

Plots the calibration curves, the ECEs and the marginal histograms of predicted posterior model probabilities for a model comparison problem. The marginal histograms inform about the fraction of predictions in each bin. Depends on the expected_calibration_error function for computing the ECE.

Parameters:

pred_modelsnp.ndarray of shape (num_data_sets, num_models): The predicted posterior model probabilities (PMPs) per data set.
true_modelsnp.ndarray of shape (num_data_sets, num_models): The one-hot-encoded true model indices per data set.
model_nameslist or None, optional, default: None: The model names for nice plot titles. Inferred if None.
num_binsint, optional, default: 10: The number of bins to use for the calibration curves (and marginal histograms).
label_fontsizeint, optional, default: 16: The font size of the y-label and y-label texts
metric_fontsizeint, optional, default: 14: The font size of the metric (e.g., ECE)
title_fontsizeint, optional, default: 18: The font size of the title text. Only relevant if stacked=False
tick_fontsizeint, optional, default: 12: The font size of the axis ticklabels
epsilonfloat, optional, default: 0.02: A small amount to pad the [0, 1]-bounded axes from both side.
figsizetuple or None, optional, default: None: The figure size passed to the matplotlib constructor. Inferred if None
colorstr, optional, default: ‘#8f2727’: The color of the calibration curves
num_rowint, optional, default: None: The number of rows for the subplots. Dynamically determined if None.
num_colint, optional, default: None: The number of columns for the subplots. Dynamically determined if None.

Returns:

figplt.Figure - the figure instance for optional saving