root_mean_squared_error#

bayesflow.diagnostics.metrics.root_mean_squared_error(estimates: Mapping[str, ~numpy.ndarray] | ~numpy.ndarray, targets: Mapping[str, ~numpy.ndarray] | ~numpy.ndarray, variable_keys: Sequence[str] = None, variable_names: Sequence[str] = None, test_quantities: dict[str, ~collections.abc.Callable]=None, normalize: str | None = 'prior', aggregation: Callable = <function median>) → dict[str, any][source]#

Computes the (Normalized) Root Mean Squared Error (RMSE/NRMSE) for the given posterior and prior samples. The values of the default normalization (prior) should be interpreted as 0 indicating the most informative (posterior is a point mass at ground truth) and 1 indicating non-informative (posterior equals prior) results.

Parameters:
estimatesnp.ndarray or dict[str, np.ndarray]

Posterior samples, either as a NumPy array of shape (num_datasets, num_draws_post, num_variables) or as a dictionary mapping variable names to arrays. Comprises num_draws_post random draws from the posterior distribution for each data set from num_datasets.

targetsnp.ndarray or dict[str, np.ndarray]

Prior samples, either as a NumPy array of shape (num_datasets, num_variables) or as a dictionary mapping variable names to arrays. Comprises num_datasets ground truths.

variable_keysSequence[str], optional (default = None)

Select keys from the dictionaries provided in estimates and targets. By default, select all keys.

variable_namesSequence[str], optional (default = None)

Optional variable names to show in the output.

test_quantitiesdict or None, optional, default: None

A dict that maps plot titles to functions that compute test quantities based on estimate/target draws.

The dict keys are automatically added to variable_keys and variable_names. Test quantity functions are expected to accept a dict of draws with shape (batch_size, ...) as the first (typically only) positional argument and return an NumPy array of shape (batch_size,). The functions do not have to deal with an additional sample dimension, as appropriate reshaping is done internally.

normalizestr or None, optional (default = “prior”)

Whether to normalize the RMSE using statistics of the prior samples. Possible options are (“mean”, “range”, “median”, “iqr”, “std”, “prior”, None)

aggregationCallable, optional (default = np.median)

Function to aggregate the RMSE across draws. Typically np.mean or np.median.

Returns:
resultdict

Dictionary containing:

  • “values”np.ndarray

    The aggregated (N)RMSE for each variable.

  • “metric_name”str

    The name of the metric (“RMSE” or “NRMSE”).

  • “variable_names”str

    The (inferred) variable names.

Notes

Aggregation is performed after computing the RMSE for each posterior draw, instead of first aggregating the posterior draws and then computing the RMSE between aggregates and ground truths.