ModelComparisonWorkflow#
- class bayesflow.workflows.ModelComparisonWorkflow(simulator: Sequence[Simulator] | ModelComparisonSimulator | None = None, adapter: Adapter | None = None, subnet: Layer | str = 'mlp', summary_network: SummaryNetwork | str | None = None, scoring_rules: CategoricalScoringRule | Sequence[CategoricalScoringRule | str] | dict[str, CategoricalScoringRule] | str = 'cross_entropy', initial_learning_rate: float = 0.0005, optimizer: Optimizer | type | None = None, checkpoint_filepath: str | None = None, checkpoint_name: str = 'model', save_weights_only: bool = False, save_best_only: bool = False, inference_conditions: Sequence[str] | str | None = None, summary_variables: Sequence[str] | str | None = None, shared_simulator: Simulator | Callable | None = None, use_mixed_batches: bool = True, model_names: Sequence[str] | None = None, standardize: Sequence[str] | str | None = 'all', **kwargs)[source]#
Bases:
BasicWorkflowThis class extends
BasicWorkflowto support amortized Bayesian model comparison with scoring rules.- Parameters:
- simulatorSequence[Simulator] or ModelComparisonSimulator, optional
Either a list of per-model
Simulatorinstances, which will be automatically wrapped in aModelComparisonSimulatorwith uniform model priors, or a pre-builtModelComparisonSimulator.- adapterAdapter, optional
Adapter for data processing. If not provided, a default adapter is built that maps
model_indicestoinference_variablesand handles anyinference_conditions/summary_variables.- subnetkeras.Layer or str, optional
The shared body of the internal
ScoringRuleNetwork, on top of which one head per scoring rule is built (default:"mlp"). Accepts a Keras layer instance or any name recognised byfind_network()(e.g."mlp").- scoring_rulesCategoricalScoringRule or Sequence or dict[str, CategoricalScoringRule] or str, optional
Scoring rule(s) used to train the classifier. Accepts, in increasing generality:
a single
CategoricalScoringRuleinstance;a string name recognised by
find_scoring_rule();a list/tuple of scores (or names) to co-learn several scores simultaneously, which are auto-named after their class (e.g.
"cross_entropy_score");a mapping of explicit names to scores.
Multiple scores share one classifier body but get separate heads and are trained jointly. Their outputs can then be kept per-rule or aggregated at estimation time via the
merge_scoresargument ofestimate(). Defaults to"cross_entropy". The chosen rule(s) determine the training dynamics. All scores produce estimates for posterior model probabilities, log odds, and potentially log Bayes factors (BFs).- summary_networkSummaryNetwork or str, optional
Optional summary network for data compression (default: None).
- initial_learning_ratefloat, optional
Initial learning rate for the optimizer (default: 5e-4).
- optimizerkeras.optimizers.Optimizer or type, optional
Optimizer instance or class. If None, a default schedule is built at fit time (default: None).
- checkpoint_filepathstr, optional
Directory for saving model checkpoints (default: None).
- checkpoint_namestr, optional
Base name for checkpoint files (default:
"model").- save_weights_onlybool, optional
Save only weights rather than the full model (default: False).
- save_best_onlybool, optional
Save only the best checkpoint instead of the last (default: False).
- inference_conditionsSequence[str] or str, optional
Keys in the simulator output to use as direct classifier conditions. When
shared_simulatoris provided, these keys are automatically broadcast to the batch dimension before being passed to the network.- summary_variablesSequence[str] or str, optional
Keys in the simulator output to compress via the summary network.
- shared_simulatorSimulator or Callable, optional
A shared simulator that provides context variables to every per-model simulator (e.g. sample size
n). Whensimulatoris a list this is forwarded toModelComparisonSimulator.- use_mixed_batchesbool, optional
Whether each training batch mixes samples from different models (default:
True). Forwarded toModelComparisonSimulatorwhensimulatoris a list.- model_namesSequence[str], optional
Human-readable names for each model, used in diagnostic plots.
- standardizeSequence[str] or str or None, optional
Variables to standardize. Defaults to all available input variables.
inference_variables(one-hot model indices) are never standardized.- **kwargsdict, optional
Additional keyword arguments organised by context:
subnet_kwargs: dict — used inScoringRuleNetworkto constructsubnetwhen it is given as a name.summary_kwargs: dict — passed tofind_summary_network().optimizer_kwargs: dict — passed to_init_optimizer.simulator_kwargs: dict — passed toModelComparisonSimulator.Other keys forwarded to
ModelComparisonApproximator.
- static default_adapter(inference_conditions: Sequence[str] | str, summary_variables: Sequence[str] | str) Adapter[source]#
Build a default adapter for model comparison data.
Maps the
model_indiceskey produced byModelComparisonSimulatortoinference_variables, and optionally handles condition and summary keys.- Parameters:
- inference_conditionsSequence[str] or str or None
Keys to concatenate into
inference_conditions.- summary_variablesSequence[str] or str or None
Keys to concatenate into
summary_variables.
- Returns:
- Adapter
A configured Adapter instance that applies dtype conversion and concatenation.
- plot_default_diagnostics(test_data: Mapping[str, ndarray] | int, estimates: Mapping[str, ndarray] | None = None, true_log_bfs_fn: Callable | None = None, **kwargs) dict[str, Figure][source]#
Generate default diagnostic plots for model comparison.
Produces a loss curve (when training history is available) followed by a set of plots that depend on the scoring rule families present. With multiple scoring rules the diagnostics run on the pooled estimates (see
estimate()), so all plots are produced for a single rule, multiple scores, and mixed families alike.Always produced:
"confusion_and_bayes_factors"— the posterior model probability confusion matrix (PMP) and the pairwise Bayes factor (BF) heatmap, drawn side by side in a single figure."calibration"— per-model calibration curves with ECE annotations."bayes_factor_recovery"— scatter of predicted vs. true log Bayes factors, one panel per competing model. Only produced whentrue_log_bfs_fnis supplied.
- Parameters:
- test_dataMapping[str, np.ndarray] or int
Either a pre-simulated data dictionary (as returned by
simulate()) or an integer specifying how many datasets to generate using the attached simulator.- estimates: Mapping[str, np.ndarray] or None
Optional pre-computed dictionary of estimates corresponding to the quantities in
test_data. Default isNone(i.e., esimtates are obtained from the test data by calling the workflow’s.estimate()method).- true_log_bfs_fncallable or None, optional
A function
(test_data: dict) -> np.ndarraythat receives the simulated data dictionary and returns ground-truth log Bayes factors of shape(num_datasets, num_models - 1). When provided, a"bayes_factor_recovery"plot is added for Bayes factor scoring rules. Ignored for PMP scoring rules.- **kwargsdict, optional
Fine-grained control over individual plots via nested dicts:
test_data_kwargs— forwarded tosimulate()whentest_datais an integer.predict_kwargs— forwarded topredict().loss_kwargs— forwarded toloss().confusion_matrix_kwargs— forwarded tomc_confusion_matrix()(PMP only).calibration_kwargs— forwarded tomc_calibration()(both PMP and BF rules).bayes_factor_recovery_kwargs— forwarded tobayes_factor_recovery()(BF scoring rules only, requirestrue_log_bfs_fn).pairwise_bayes_factors_kwargs— forwarded topairwise_bayes_factors()(BF scoring rules only).matrix_row_figsize— figure size(width, height)for the combined side-by-side confusion matrix / pairwise Bayes factor figure, used only when both scoring rule families are present. Inferred from the number of models if omitted.
- Returns:
- dict[str, plt.Figure]
Keys are plot names; values are the corresponding
Figureobjects.
- Raises:
- ValueError
If
test_datais an integer but no simulator is attached.
- compute_default_diagnostics(test_data: Mapping[str, ndarray] | int, estimates: Mapping[str, ndarray] | None = None, as_data_frame: bool = True, **kwargs) Sequence[dict] | DataFrame[source]#
Compute default scalar diagnostic metrics for model comparison.
All metrics are computed from the pooled log odds, see
estimate()):"accuracy"— per-model classification accuracy, i.e., the diagonal entries of the row-normalized confusion matrix (fraction of datasets simulated from modelmfor whichargmax(PMPs)also predicts modelm), as returned byaccuracy(). Controlled via theper_modelkeyword (defaultTrue); set toFalseto instead obtain the overall (aggregate) accuracy as a single float."ece"— per-model expected calibration errors, as returned byexpected_calibration_error()."brier_score"— per-model (one-vs-rest) and aggregate multi-class Brier scores, as returned bybrier_score().
- Parameters:
- test_dataMapping[str, np.ndarray] or int
Either a pre-simulated data dictionary or an integer specifying how many datasets to generate using the attached simulator.
- estimates: Mapping[str, np.ndarray] or None
Optional pre-computed dictionary of estimates corresponding to the quantities in
test_data. Default isNone(i.e., esimtates are obtained from the test data by calling the workflow’s.estimate()method).- as_data_framebool, optional
Whether to return the results as a pandas DataFrame (default: True), with one column per model and one row per metric. If False, a dictionary of the raw (unparsed) metric outputs is returned instead.
- **kwargsdict, optional
Fine-grained control via nested dicts:
test_data_kwargs— forwarded tosimulate()whentest_datais an integer.estimate_kwargs— forwarded toestimate().accuracy_kwargs— forwarded toaccuracy()(e.g.{"per_model": False}for the aggregate accuracy).ece_kwargs— forwarded toexpected_calibration_error().
- Returns:
- dict[str, Any] or pd.DataFrame
If
as_data_frameis True, a pandas DataFrame with one row per metric and one column per model. Otherwise, a dictionary of the raw metric outputs.
- Raises:
- ValueError
If
test_datais an integer but no simulator is attached.
- sample(*args, **kwargs)[source]#
Draws num_samples samples from the approximator given specified conditions.
- Parameters:
- num_samplesint
The number of samples to generate.
- conditionsdict[str, np.ndarray], optional
A dictionary where keys represent variable names and values are NumPy arrays containing the adapted simulated variables. Keys used as summary or inference conditions during training should be present.
- splitbool, default=False
Whether to split the output arrays along the last axis and return one sample array per target variable.
- batch_sizeint or None, optional
If provided, the conditions are split into batches of size batch_size, for which samples are generated sequentially. Can help with memory management for large sample sizes.
- sample_shapestr or tuple of int, optional
Trailing structural dimensions of each generated sample, excluding the batch and target (intrinsic) dimension. For example, use (time,) for time series or (height, width) for images.
If set to “infer” (default), the structural dimensions are inferred from the inference_conditions. In that case, all non-vector dimensions except the last (channel) dimension are treated as structural dimensions. For example, if the final inference_conditions have shape (batch_size, time, channels), then sample_shape is inferred as (time,), and the generated samples will have shape (num_conditions, num_samples, time, target_dim).
- seedint, keras.random.SeedGenerator, or None, optional
Seed for reproducible sampling. An integer is converted to a
keras.random.SeedGeneratorand shared across all stochastic operations in the call. ASeedGeneratoris passed through as-is. IfNone(default), each component uses its own instance seed generator.- **kwargsdict | str, optional
Additional keyword arguments passed to the approximator’s sampling function.
- Returns:
- dict[str, np.ndarray]
A dictionary where keys correspond to variable names and values are arrays containing the generated samples.
- log_prob(*args, **kwargs)[source]#
Compute the log probability of given variables under the approximator.
- Parameters:
- dataMapping[str, np.ndarray]
A dictionary where keys represent variable names and values are arrays corresponding to the variables’ realizations.
- **kwargsdict | str, optional
Additional keyword arguments passed to the approximator’s log probability function.
- Returns:
- np.ndarray
An array containing the log probabilities computed from the provided variables.
- ancestral_sample(*args, **kwargs)[source]#
Draws samples from the approximator given specified conditions and ancestral conditions.
- Parameters:
- conditionsdict[str, np.ndarray]
A dictionary where keys represent variable names and values are NumPy arrays containing the adapted simulated variables. Keys used as summary or inference conditions during training should be present. Should have shape (n_datasets, n_conditions, …).
- ancestral_conditionsdict[str, np.ndarray]
A dictionary where keys represent variable names and values are NumPy arrays containing the ancestral conditions for sampling. These are used in ancestral sampling scheme (e.g. a hierarchical model). Should have shape (n_datasets, n_ancestral_conditions, …).
- summariesTensor | np.ndarray | None, optional
Precomputed summary outputs to be used as conditions for sampling. If provided, these will be used instead of the conditions. Should have shape (n_datasets, n_conditions, …).
- splitbool, default=False
Whether to split the output arrays along the last axis and return one sample array per target variable.
- batch_sizeint or None, optional
If provided, the conditions are split into batches of size batch_size, for which samples are generated sequentially. Can help with memory management for large sample sizes.
- sample_shapestr or tuple of int, optional
Trailing structural dimensions of each generated sample, excluding the batch and target (intrinsic) dimension. For example, use (time,) for time series or (height, width) for images.
If set to “infer” (default), the structural dimensions are inferred from the inference_conditions. In that case, all non-vector dimensions except the last (channel) dimension are treated as structural dimensions. For example, if the final inference_conditions have shape (batch_size, time, channels), then sample_shape is inferred as (time,), and the generated samples will have shape (num_conditions, num_samples, time, target_dim).
- **kwargsdict | str, optional
Additional keyword arguments passed to the approximator’s sampling function.
- Returns:
- dict[str, np.ndarray]
A dictionary where keys correspond to variable names and values are arrays containing the generated samples.
- property adapter#
- build_graph(*args, **kwargs)#
- build_optimizer(epochs: int, num_batches: int, strategy: str) Optimizer | None#
Build and initialize the optimizer based on the training strategy. Uses a cosine decay learning rate schedule, where the final learning rate is proportional to the square of the initial learning rate, as found to work best in SBI.
The default optimizer will use 5% of the epochs as warmup; during the warmup phase, the learning rate will be increased from 10% of the initial learning rate to initial learning rate supplied to the workflow.
- Parameters:
- epochsint
The total number of training epochs.
- num_batchesint
The number of batches per epoch.
- strategystr
The training strategy, either “online” or another mode that applies weight decay. For “online” training, an Adam optimizer with gradient clipping is used. For other strategies, AdamW is used with weight decay to encourage regularization.
- Returns:
- keras.Optimizer or None
The initialized optimizer if it was not already set. Returns None if the optimizer was already defined.
- compute_custom_diagnostics(test_data: Mapping[str, ndarray] | int, metrics: Mapping[str, Callable], num_samples: int = 1000, samples: Mapping[str, ndarray] | None = None, variable_keys: Sequence[str] | None = None, variable_names: Sequence[str] | None = None, as_data_frame: bool = True, **kwargs) Sequence[Mapping] | DataFrame#
Computes custom diagnostic metrics to evaluate the quality of inference. The metric functions should have a signature of:
metric_fn(samples, inference_variables, variable_names, variable_keys) or
metric_fn(samples, inference_variables, **kwargs)
The functions should return a dictionary containing the metric name in
metric_namekey and the metric values in avalueskey.- Parameters:
- test_dataMapping[str, np.ndarray] or int
A dictionary containing test data where keys represent variable names and values are corresponding realizations. If an integer is provided, that number of test data sets will be generated using the simulator (if available).
- metrics: Mapping[str, Callable]
A dictionary containing custom metric functions where keys represent the function names and values correspond to the functions themselves. The functions should have a signature of fn(samples, inference_variables, variable_names)
- num_samplesint, optional
The number of samples to draw from the approximator for diagnostics, by default 1000.
- samplesMapping[str, array], optional
Pre-computed samples from workflow.sample or approximator.sample. If provided, the num_samples argument is ignored. Providing samples requires you to also provide the test_data used to obtain the samples.
- variable_keyslist or None, optional, default: None
Select keys from the dictionaries provided in estimates and targets. By default, select all keys.
- variable_nameslist or None, optional, default: None
The variable names for nice plot titles.
- as_data_framebool, optional
Whether to return the results as a pandas DataFrame (default: True). If False, a sequence of dictionaries with metric values is returned.
- **kwargsdict, optional
Additional keyword arguments:
- test_data_kwargs: dict, optional
Arguments to pass to the simulator when generating test data.
- approximator_kwargs: dict, optional
Arguments to pass to the approximator’s sampling function.
- root_mean_squared_error_kwargs: dict, optional
Arguments for customizing the RMSE computation.
- posterior_contraction_kwargs: dict, optional
Arguments for customizing the posterior contraction computation.
- calibration_error_kwargs: dict, optional
Arguments for customizing the calibration error computation.
- Returns:
- Sequence[dict] or pd.DataFrame
If as_data_frame is True, returns a pandas DataFrame containing the computed diagnostic metrics for each variable. Otherwise, returns a sequence of dictionaries with metric values.
- compute_diagnostics(**kwargs)#
- estimate(*, conditions: Mapping[str, ndarray] | None = None, **kwargs) dict[str, dict[str, ndarray | dict[str, ndarray]]]#
Estimates point summaries of inference variables based on specified conditions.
- Parameters:
- conditionsMapping[str, np.ndarray], optional
A dictionary mapping variable names to arrays representing the conditions for the estimation process.
- **kwargsdict | str
Additional keyword arguments passed to underlying processing functions.
- Returns:
- estimatesdict[str, dict[str, np.ndarray or dict[str, np.ndarray]]]
The estimates of inference variables in a nested dictionary.
Each first-level key is the name of an inference variable.
Each second-level key is the name of a scoring rule.
(If the scoring rule comprises multiple estimators, each third-level key is the name of an estimator.)
Each estimator output (i.e., dictionary value that is not itself a dictionary) is an array of shape (num_datasets, point_estimate_size, variable_block_size).
- fit(**kwargs)#
- fit_disk(root: PathLike, pattern: str = '*.pkl', batch_size: int = 32, load_fn: Callable | None = None, epochs: int = 100, keep_optimizer: bool = False, validation_data: Mapping[str, ndarray] | int | None = None, augmentations: Mapping[str, Callable] | Callable | None = None, **kwargs) History#
Train the approximator using data stored on disk. This approach is suitable for large sets of simulations that don’t fit in memory.
- Parameters:
- rootos.PathLike
The root directory containing the dataset files.
- patternstr, optional
A filename pattern to match dataset files, by default
"*.pkl".- batch_sizeint, optional
The batch size used for training, by default 32.
- load_fnCallable, optional
A function to load dataset files. If None, a default loading function is used.
- epochsint, optional
The number of training epochs, by default 100. Consider increasing this number for free-form inference networks, such as FlowMatching or ConsistencyModel, which generally need more epochs than CouplingFlows.
- keep_optimizerbool, optional
Whether to retain the current state of the optimizer after training, by default False.
- validation_dataMapping[str, np.ndarray] or int, optional
A dictionary containing validation data. If an integer is provided, that number of validation samples will be generated (if supported). By default, no validation data is used.
- augmentationsdict of str to Callable or Callable, optional
Dictionary of augmentation functions to apply to each corresponding key in the batch or a function to apply to the entire batch (possibly adding new keys).
If you provide a dictionary of functions, each function should accept one element of your output batch and return the corresponding transformed element. Otherwise, your function should accept the entire dictionary output and return a dictionary.
Note - augmentations are applied before the adapter is called and are generally transforms that you only want to apply during training.
- **kwargsdict, optional
Additional keyword arguments passed to the underlying _fit method.
- Returns:
- historykeras.callbacks.History
A history object containing training history, where keys correspond to logged metrics (e.g., loss values) and values are arrays tracking metric evolution over epochs.
- fit_offline(data: Mapping[str, ndarray], epochs: int = 100, batch_size: int = 32, keep_optimizer: bool = False, validation_data: Mapping[str, ndarray] | int | None = None, augmentations: Mapping[str, Callable] | Callable | None = None, **kwargs) History#
Train the approximator offline using a fixed dataset. This approach will be faster than online training, since no computation time is spent in generating new data for each batch, but it assumes that simulations can fit in memory.
- Parameters:
- dataMapping[str, np.ndarray]
A dictionary containing training data where keys represent variable names and values are corresponding realizations.
- epochsint, optional
The number of training epochs, by default 100. Consider increasing this number for free-form inference networks, such as FlowMatching or ConsistencyModel, which generally need more epochs than CouplingFlows.
- batch_sizeint, optional
The batch size used for training, by default 32.
- keep_optimizerbool, optional
Whether to retain the current state of the optimizer after training, by default False.
- validation_dataMapping[str, np.ndarray] or int, optional
A dictionary containing validation data. If an integer is provided, that number of validation samples will be generated (if supported). By default, no validation data is used.
- augmentationsdict of str to Callable or Callable, optional
Dictionary of augmentation functions to apply to each corresponding key in the batch or a function to apply to the entire batch (possibly adding new keys).
If you provide a dictionary of functions, each function should accept one element of your output batch and return the corresponding transformed element. Otherwise, your function should accept the entire dictionary output and return a dictionary.
Note - augmentations are applied before the adapter is called and are generally transforms that you only want to apply during training.
- **kwargsdict, optional
Additional keyword arguments passed to the underlying _fit method.
- Returns:
- historykeras.callbacks.History
A history object containing training history, where keys correspond to logged metrics (e.g., loss values) and values are arrays tracking metric evolution over epochs.
- fit_online(epochs: int = 100, num_batches_per_epoch: int = 100, batch_size: int = 32, keep_optimizer: bool = False, validation_data: Mapping[str, ndarray] | int | None = None, augmentations: Mapping[str, Callable] | Callable | None = None, **kwargs) History#
Train the approximator using an online data-generating process. The dataset is dynamically generated during training, making this approach suitable for scenarios where generating new simulations is computationally cheap.
- Parameters:
- epochsint, optional
The number of training epochs, by default 100.
- num_batches_per_epochint, optional
The number of batches generated per epoch, by default 100.
- batch_sizeint, optional
The batch size used for training, by default 32.
- keep_optimizerbool, optional
Whether to retain the current state of the optimizer after training, by default False.
- validation_dataMapping[str, np.ndarray] or int, optional
A dictionary containing validation data. If an integer is provided, that number of validation samples will be generated (if supported). By default, no validation data is used.
- augmentationsdict of str to Callable or Callable, optional
Dictionary of augmentation functions to apply to each corresponding key in the batch or a function to apply to the entire batch (possibly adding new keys).
If you provide a dictionary of functions, each function should accept one element of your output batch and return the corresponding transformed element. Otherwise, your function should accept the entire dictionary output and return a dictionary.
Note - augmentations are applied before the adapter is called and are generally transforms that you only want to apply during training.
- **kwargsdict, optional
Additional keyword arguments passed to the underlying _fit method.
- Returns:
- historykeras.callbacks.History
A history object containing training history, where keys correspond to logged metrics (e.g., loss values) and values are arrays tracking metric evolution over epochs.
- property inference_network#
- load_approximator(path: str | PathLike | None = None)#
Restore the approximator from a saved checkpoint.
When
pathisNone, the checkpoint location is derived from the workflow’scheckpoint_filepathandcheckpoint_nameattributes. The expected filename is<checkpoint_name>.weights.h5whensave_weights_only=Truewas set, and<checkpoint_name>.kerasfor a fully serialized model.For weights-only checkpoints (
.weights.h5), the current approximator’s architecture is kept and only the weights are replaced viaapproximator.load_weights(). For full-model checkpoints (.keras), the entire approximator object is replaced viakeras.saving.load_model().- Parameters:
- pathstr or os.PathLike, optional
Explicit path to the checkpoint file. The file extension determines the loading strategy:
*.weights.h5→ weights-only restore (load_weights).any other extension (e.g.
*.keras) -> full model restore (keras.saving.load_model).
If
None(default), the path is inferred fromcheckpoint_filepath/checkpoint_name.
- Raises:
- ValueError
If
pathisNoneandcheckpoint_filepathis not set on the workflow.- FileNotFoundError
If the resolved checkpoint path does not exist on disk.
Notes
Weights-only restore requires the model to be built first. BayesFlow adapters record internal state during the first
strict=Trueforward pass, which only happens infit. Callworkflow.fit_online(...)(orfit_offline/fit_disk) once before loading a.weights.h5checkpoint. The default.kerasformat avoids this requirement entirely.
- plot_custom_diagnostics(test_data: Mapping[str, ndarray] | int, plot_fns: Mapping[str, Callable], num_samples: int = 1000, samples: Mapping[str, ndarray] | None = None, variable_keys: Sequence[str] | None = None, variable_names: Sequence[str] | None = None, **kwargs) dict[str, Figure]#
Generates custom diagnostic plots to evaluate the quality of inference. The functions passed should have the following signature: - fn(samples, inference_variables, variable_names)
They should also return a single matplotlib Figure object.
- Parameters:
- test_dataMapping[str, np.ndarray] or int
A dictionary containing test data where keys represent variable names and values are corresponding data arrays. If an integer is provided, that number of test data sets will be generated using the simulator (if available).
- plot_fns: Mapping[str, Callable]
A dictionary containing custom plotting functions where keys represent the function names and values correspond to the functions themselves. The functions should have a signature of fn(samples, inference_variables, variable_names)
- num_samplesint, optional
The number of samples to draw from the approximator for diagnostics, by default 1000.
- samplesMapping[str, array], optional
Pre-computed samples from workflow.sample or approximator.sample. If provided, the num_samples argument is ignored. Providing samples requires you to also provide the test_data used to obtain the samples.
- variable_keyslist or None, optional, default: None
Select keys from the dictionaries provided in estimates and targets. By default, select all keys.
- variable_nameslist or None, optional, default: None
The variable names for nice table plot titles.
- **kwargsdict, optional
Additional keyword arguments:
- test_data_kwargs: dict, optional
Arguments to pass to the simulator when generating test data.
- approximator_kwargs: dict, optional
Arguments to pass to the approximator’s sampling function.
- loss_kwargs: dict, optional
Arguments for customizing the loss plot.
- recovery_kwargs: dict, optional
Arguments for customizing the parameter recovery plot.
- calibration_ecdf_kwargs: dict, optional
Arguments for customizing the empirical cumulative distribution function (ECDF) calibration plot.
- z_score_contraction_kwargs: dict, optional
Arguments for customizing the z-score contraction plot.
- Returns:
- dict[str, plt.Figure]
A dictionary where keys correspond to different diagnostic plot types, and values are the respective matplotlib Figure objects.
- plot_diagnostics(**kwargs)#
- static samples_to_data_frame(samples: Mapping[str, ndarray]) DataFrame#
Convert a dictionary of samples into a pandas DataFrame.
- Parameters:
- samplesMapping[str, np.ndarray]
A dictionary where keys represent variable names and values are arrays containing sampled data.
- Returns:
- pd.DataFrame
A pandas DataFrame where each column corresponds to a variable, and rows represent individual samples.
- simulate(batch_shape: tuple[int, ...], **kwargs) dict[str, ndarray]#
Generates a batch of simulations using the provided simulator.
- Parameters:
- batch_shapeShape
The shape of the batch to be simulated. Typically, an integer for simple simulators.
- **kwargsdict | str, optional
Additional keyword arguments passed to the simulator’s sample method.
- Returns:
- dict[str, np.ndarray]
A dictionary where keys represent variable names and values are NumPy arrays containing the simulated variables.
- Raises:
- RuntimeError
If no simulator is provided.
- simulate_adapted(batch_shape: tuple[int, ...], **kwargs) dict[str, ndarray]#
Generates a batch of simulations and applies the adapter to the result.
- Parameters:
- batch_shapeShape
The shape of the batch to be simulated. Typically, an integer for simple simulators.
- **kwargsdict | str, optional
Additional keyword arguments passed to the simulator’s sample method.
- Returns:
- dict[str, np.ndarray]
A dictionary where keys represent variable names and values are NumPy arrays containing the adapted simulated variables.
- Raises:
- RuntimeError
If no simulator is provided.
- property summary_network#