3. Data Processing: Adapters#

Disclaimer: This guide is in an early stage. We welcome contributions to the guide in form of issues and pull requests.

To ensure that the training data generated by a simulator can be used for deep learning, we have to bring our data into the structure required by BayesFlow. The Adapter class provides multiple flexible functionalities, from standardization to renaming, and many more.

3.1. BayesFlow’s Data Structure#

BayesFlow offers a standardized interface for training neural networks. Data and parameters are organized in dictionaries. The inputs to the networks are organized in specific dictionary entries.

inference_variables (required): The variables of the distribution we try to approximate. For a posterior distribution, this would be the parameters. For a likelihood function, this would be the data.
summary_variables (optional): Variables that are passed through the summary network, and subsequently used as a condition for the inference network. In a posterior estimation setting, this would be the data (if a summary network is used).
inference_conditions (optional): Conditions for the inference network that are passed directly, without going through a summary network. This is useful for context variables, as well as for the data when not summary network is used.

In addition, we have to ensure that the correct data type is passed, usually float32. The Adapter class makes it easy to transform the data into the required structure.

3.1.1. Example: Posterior Estimation#

Let’s start with a simple posterior estimation example, where we want to approximate the posterior distribution for parameters theta_1 and theta_2, conditional on data x. First, we construct a simple dataset.

import bayesflow as bf
import numpy as np

batch_size = 2
rng = np.random.default_rng(seed=2025)
data = {
    "theta_1": np.zeros((batch_size, 1)),
    "theta_2": np.ones((batch_size, 1)),
    "x": rng.uniform(size=(batch_size, 3)),
}
print("Shapes:", {k: v.shape for k, v in data.items()})

Shapes: {'theta_1': (2, 1), 'theta_2': (2, 1), 'x': (2, 3)}

Next, we create an Adapter to convert it into the desired format (assuming we want to use a summary network later on).

adapter = (
    bf.Adapter()
    .convert_dtype("float64", "float32")
    .concatenate(["theta_1", "theta_2"], into="inference_variables")
    .rename("x", "summary_variables")
)

print(adapter)

Adapter([0: ConvertDType -> 1: Concatenate(['theta_1', 'theta_2'] -> 'inference_variables') -> 2: Rename('x' -> 'summary_variables')])

When we now apply the adapter to our data, it executes the specified transformations:

transformed_data = adapter(data)
print(transformed_data)
print("Shapes:", {k: v.shape for k, v in transformed_data.items()})

{'inference_variables': array([[0., 1.],
       [0., 1.]], dtype=float32), 'summary_variables': array([[0.9944578 , 0.38200974, 0.827148  ],
       [0.8372553 , 0.97580904, 0.07722503]], dtype=float32)}
Shapes: {'inference_variables': (2, 2), 'summary_variables': (2, 3)}

Many of the transforms in the adapter are invertible, so that we can also call the adapter in the inverse direction:

cycled_data = adapter(transformed_data, inverse=True)
print("Shapes:", {k: v.shape for k, v in cycled_data.items()})

Shapes: {'x': (2, 3), 'theta_1': (2, 1), 'theta_2': (2, 1)}

3.1.2. Example: Likelihood Estimation#

For likelihood estimation, the roles are switched. We want to estimate the distribution of the data x conditional on the parameters theta_1 and theta_2. We supply the parameters to the inference network directly without a summary network.

adapter = (
    bf.Adapter()
    .convert_dtype("float64", "float32")
    .concatenate(["theta_1", "theta_2"], into="inference_conditions")
    .rename("x", "inference_variables")
)

print(adapter)
transformed_data = adapter(data)
print("Shapes:", {k: v.shape for k, v in transformed_data.items()})

Adapter([0: ConvertDType -> 1: Concatenate(['theta_1', 'theta_2'] -> 'inference_conditions') -> 2: Rename('x' -> 'inference_variables')])
Shapes: {'inference_conditions': (2, 2), 'inference_variables': (2, 3)}

You can find many more configurations in the Examples section.

3.2. Pre-processing#

Besides the structure and the data types, there are pre-processing steps that can make network training more efficient. Those include standardization, transforming constrained variables to an unconstrained space, or various non-linear transformations that simply the space the network has to operate in. In addition, operations on arrays like broadcasting and concatenating simplify the transformation into the required structure.

The Adapter features a large set of methods, please refer to the API documentation for a complete list. For applied examples, refer to the Examples section.