calibration_histogram

bayesflow.diagnostics.calibration_histogram(estimates: Mapping[str, ndarray] | ndarray, targets: Mapping[str, ndarray] | ndarray, variable_keys: Sequence[str] = None, variable_names: Sequence[str] = None, test_quantities: dict[str, Callable] = None, figsize: Sequence[float] = None, num_bins: int = 10, binomial_interval: float = 0.99, label_fontsize: int = 16, title_fontsize: int = 18, tick_fontsize: int = 12, color: str = '#132a70', num_col: int = None, num_row: int = None) → Figure

Creates and plots publication-ready histograms of rank statistics for simulation-based calibration (SBC) checks according to [1].

Any deviation from uniformity indicates miscalibration and thus poor convergence of the networks or poor combination between generative model / networks.

[1] Talts, S., Betancourt, M., Simpson, D., Vehtari, A., & Gelman, A. (2018). Validating Bayesian inference algorithms with simulation-based calibration. arXiv preprint arXiv:1804.06788.

Parameters:

estimatesnp.ndarray of shape (n_data_sets, n_post_draws, n_params)

The posterior draws obtained from n_data_sets

targetsnp.ndarray of shape (n_data_sets, n_params)

The prior draws obtained for generating n_data_sets

variable_keyslist or None, optional, default: None

Select keys from the dictionaries provided in estimates and targets. By default, select all keys.

variable_nameslist or None, optional, default: None

The parameter names for nice plot titles. Inferred if None

test_quantitiesdict or None, optional, default: None

A dict that maps plot titles to functions that compute test quantities based on estimate/target draws.

The dict keys are automatically added to variable_keys and variable_names. Test quantity functions are expected to accept a dict of draws with shape (batch_size, ...) as the first (typically only) positional argument and return an NumPy array of shape (batch_size,). The functions do not have to deal with an additional sample dimension, as appropriate reshaping is done internally.

figsizetuple or None, optional, defaultNone

The figure size passed to the matplotlib constructor. Inferred if None

num_binsint, optional, default: 10

The number of bins to use for each marginal histogram

binomial_intervalfloat in (0, 1), optional, default: 0.99

The width of the confidence interval for the binomial distribution

label_fontsizeint, optional, default: 16

The font size of the y-label text

title_fontsizeint, optional, default: 18

The font size of the title text

tick_fontsizeint, optional, default: 12

The font size of the axis ticklabels

colorstr, optional, default ‘#a34f4f’

The color to use for the histogram body

num_rowint, optional, default: None

The number of rows for the subplots. Dynamically determined if None.

num_colint, optional, default: None

The number of columns for the subplots. Dynamically determined if None.

Returns:

fplt.Figure - the figure instance for optional saving

Raises:

ShapeError

If there is a deviation form the expected shapes of estimates and targets.