simultaneous_ecdf_bands#
- bayesflow.utils.simultaneous_ecdf_bands(num_estimates: int, num_points: int = None, num_simulations: int = 1000, confidence: float = 0.95, eps: float = 1e-05, max_num_points: int = 1000) Sequence[source]#
Computes the simultaneous ECDF bands through simulation according to the algorithm described in Section 2.2 of
Säilynoja, T., Bürkner, P. C., & Vehtari, A. (2022). Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison. Statistics and Computing, 32(2), 32. See: https://link.springer.com/article/10.1007/s11222-022-10090-6
Depends on the vectorized utility function ecdf.minimal_coverage_probs(z, u). Will be used by the diagnostics module to create the ECDF marginal calibration plots.
- Parameters:
- num_estimatesint
The sample size used for computing the ECDF. Will equal to the number of simulated conditions when used for simulation-based calibration. Corresponds to N in the paper above.
- num_pointsint, optional, default: None
The number of evaluation points on the interval (0, 1). Defaults to num_points = num_estimates if not explicitly specified. Correspond to K in the paper above.
- num_simulationsint, optional, default: 1000
The number of samples of size n_samples to simulate for determining the simultaneous CIs.
- confidencefloat in (0, 1), optional, default: 0.95
The confidence level, confidence = 1 - alpha specifies the width of the confidence interval.
- epsfloat, optional, default: 1e-5
Small number to add to the lower and subtract from the upper bound of the interval [0, 1] to avoid edge artefacts. No need to touch this.
- max_num_pointsint, optional, default: 1000
Upper bound on num_points. Saves computation time when num_estimates is large.
- Returns:
- (alpha, z, L, U) - tuple of scalar and three arrays of size (num_estimates,) containing the confidence level
as well as the evaluation points, the lower, and the upper confidence bands, respectively.