bayesflow.helper_networks module

class bayesflow.helper_networks.DenseCouplingNet(*args, **kwargs)

Bases: Model

Implements a conditional version of a standard fully connected (FC) network. Would also work as an unconditional estimator.

__init__(settings, dim_out, **kwargs)

Creates a conditional coupling net (FC neural network).

Parameters:

settingsdict

A dictionary holding arguments for a dense layer: See https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense

As well as custom arguments for settings such as residual networks, dropout, and spectral normalization.

dim_outint

Number of outputs of the coupling net. Determined internally by the consumer classes.

**kwargsdict, optional, default: {}

Optional keyword arguments passed to the tf.keras.Model constructor.

call(target, condition, **kwargs)

Concatenates target and condition and performs a forward pass through the coupling net.

Parameters:

targettf.Tensor

The split estimation quntities, for instance, parameters \(\theta \sim p(\theta)\) of interest, shape (batch_size, …)

conditiontf.Tensor or None

the conditioning vector of interest, for instance x = summary(x), shape (batch_size, summary_dim)

class bayesflow.helper_networks.Permutation(*args, **kwargs)

Bases: Model

Implements a layer to permute the inputs entering a (conditional) coupling layer. Uses fixed permutations, as these perform equally well compared to learned permutations.

__init__(input_dim)

Creates an invertible permutation layer for a conditional invertible layer.

Parameters:

input_dimint

Ihe dimensionality of the input to the (conditional) coupling layer.

call(target, inverse=False)

Permutes a batch of target vectors over the last axis.

Parameters:

targettf.Tensor of shape (batch_size, …)

The target vector to be permuted over its last axis.

inversebool, optional, default: False

Controls if the current pass is forward (inverse=False) or inverse (inverse=True).

Returns:

outtf.Tensor of the same shape as target.

The (un-)permuted target vector.

class bayesflow.helper_networks.Orthogonal(*args, **kwargs)

Bases: Model

Implements a learnable orthogonal transformation according to [1]. Can be used as an alternative to a fixed Permutation layer.

[1] Kingma, D. P., & Dhariwal, P. (2018). Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31.

__init__(input_dim)

Creates an invertible orthogonal transformation (generalized permutation)

Parameters:

input_dimint

Ihe dimensionality of the input to the (conditional) coupling layer.

call(target, inverse=False)

Transforms a batch of target vectors over the last axis through an approximately orthogonal transform.

Parameters:

targettf.Tensor of shape (batch_size, …)

The target vector to be rotated over its last axis.

inversebool, optional, default: False

Controls if the current pass is forward (inverse=False) or inverse (inverse=True).

Returns:

outtf.Tensor of the same shape as target.

The (un-)rotated target vector.

class bayesflow.helper_networks.MCDropout(*args, **kwargs)

Bases: Model

Implements Monte Carlo Dropout as a Bayesian approximation according to [1].

Perhaps not the best approximation, but arguably the cheapest one out there!

[1] Gal, Y., & Ghahramani, Z. (2016, June). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning (pp. 1050-1059). PMLR.

__init__(dropout_prob=0.1, **kwargs)

Creates a custom instance of an MC Dropout layer. Will be used both during training and inference.

Parameters:

dropout_probfloat, optional, default: 0.1

The dropout rate to be passed to tf.keras.layers.Dropout().

call(inputs)

Randomly sets elements of inputs to zero.

Parameters:

inputstf.Tensor

Input of shape (batch_size, …)

Returns:

outtf.Tensor

Output of shape (batch_size, …), same as inputs.

class bayesflow.helper_networks.ActNorm(*args, **kwargs)

Bases: Model

Implements an Activation Normalization (ActNorm) Layer. Activation Normalization is learned invertible normalization, using a Scale (s) and Bias (b) vector:

y = s * x + b (forward)
x = (y - b)/s (inverse)

Notes

The scale and bias can be data dependent initialized, such that the output has a mean of zero and standard deviation of one [R2ce218db8fb3-1]_[R2ce218db8fb3-2]_. Alternatively, it is initialized with vectors of ones (scale) and zeros (bias).

References

[1]

Kingma, Diederik P., and Prafulla Dhariwal. “Glow: Generative flow with invertible 1x1 convolutions.” arXiv preprint arXiv:1807.03039 (2018).

[2]

Salimans, Tim, and Durk P. Kingma. “Weight normalization: A simple reparameterization to accelerate training of deep neural networks.” Advances in neural information processing systems 29 (2016): 901-909.

__init__(latent_dim, act_norm_init, **kwargs)

Creates an instance of an ActNorm Layer as proposed by [1].

Parameters:

latent_dimint

The dimensionality of the latent space (equal to the dimensionality of the target variable)

act_norm_initnp.ndarray of shape (num_simulations, num_params) or None, optional, default: None

Optional data-dependent initialization for the internal ActNorm layers, as done in [1]. Could be helpful for deep invertible networks.

call(target, inverse=False)

Performs one pass through the actnorm layer (either inverse or forward) and normalizes the last axis of target.

Parameters:

targettf.Tensor of shape (batch_size, …)

the target variables of interest, i.e., parameters for posterior estimation

inversebool, optional, default: False

Flag indicating whether to run the block forward or backwards

Returns:

(z, log_det_J)tuple(tf.Tensor, tf.Tensor)

If inverse=False: The transformed input and the corresponding Jacobian of the transformation, v shape: (batch_size, inp_dim), log_det_J shape: (,)

targettf.Tensor

If inverse=True: The inversely transformed targets, shape == target.shape

Notes

If inverse=False, the return is (z, log_det_J).

If inverse=True, the return is target.

class bayesflow.helper_networks.InvariantModule(*args, **kwargs)

Bases: Model

Implements an invariant module performing a permutation-invariant transform.

For details and rationale, see:

[1] Bloem-Reddy, B., & Teh, Y. W. (2020). Probabilistic Symmetries and Invariant Neural Networks. J. Mach. Learn. Res., 21, 90-1. https://www.jmlr.org/papers/volume21/19-322/19-322.pdf

__init__(settings, **kwargs)

Creates an invariant module according to [1] which represents a learnable permutation-invariant function with an option for learnable pooling.

Parameters:

settingsdict

A dictionary holding the configuration settings for the module.

**kwargsdict, optional, default: {}

Optional keyword arguments passed to the tf.keras.Model constructor.

call(x, **kwargs)

Performs the forward pass of a learnable invariant transform.

Parameters:

xtf.Tensor

Input of shape (batch_size,…, x_dim)

Returns:

outtf.Tensor

Output of shape (batch_size,…, out_dim)

class bayesflow.helper_networks.EquivariantModule(*args, **kwargs)

Bases: Model

Implements an equivariant module performing an equivariant transform.

For details and justification, see:

[1] Bloem-Reddy, B., & Teh, Y. W. (2020). Probabilistic Symmetries and Invariant Neural Networks. J. Mach. Learn. Res., 21, 90-1. https://www.jmlr.org/papers/volume21/19-322/19-322.pdf

__init__(settings, **kwargs)

Creates an equivariant module according to [1] which combines equivariant transforms with nested invariant transforms, thereby enabling interactions between set members.

Parameters:

settingsdict

A dictionary holding the configuration settings for the module.

**kwargsdict, optional, default: {}

Optional keyword arguments passed to the tf.keras.Model constructor.

call(x, **kwargs)

Performs the forward pass of a learnable equivariant transform.

Parameters:

xtf.Tensor

Input of shape (batch_size, …, x_dim)

Returns:

outtf.Tensor

Output of shape (batch_size, …, equiv_dim)

class bayesflow.helper_networks.MultiConv1D(*args, **kwargs)

Bases: Model

Implements an inception-inspired 1D convolutional layer using different kernel sizes.

__init__(settings, **kwargs)

Creates an inception-like Conv1D layer

Parameters:

settingsdict

A dictionary which holds the arguments for the internal Conv1D layers.

call(x, **kwargs)

Performs a forward pass through the layer.

Parameters:

xtf.Tensor

Input of shape (batch_size, n_time_steps, n_time_series)

Returns:

outtf.Tensor

Output of shape (batch_size, n_time_steps, n_filters)

class bayesflow.helper_networks.ConfigurableMLP(*args, **kwargs)

Bases: Model

Implements a simple configurable MLP with optional residual connections and dropout.

__init__(input_dim, hidden_dim=512, output_dim=None, num_hidden=2, activation='relu', residual=True, dropout_rate=0.05, **kwargs)

Creates an instance of a flexible and simple MLP with optional residual connections and dropout.

call(inputs, **kwargs)

Calls the model on new inputs and returns the outputs as tensors.

In this case call() just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Note: This method should not be called directly. It is only meant to be overridden when subclassing tf.keras.Model. To call a model on an input, always use the __call__() method, i.e. model(inputs), which relies on the underlying call() method.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors. training: Boolean or boolean scalar tensor, indicating whether to

run the Network in training mode or inference mode.

mask: A mask or list of masks. A mask can be either a boolean tensor

or None (no mask). For more details, check the guide [here](https://www.tensorflow.org/guide/keras/masking_and_padding).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

class bayesflow.helper_networks.ConfigurableHiddenBlock(*args, **kwargs)

Bases: Model

__init__(num_units, activation='relu', residual=True, dropout_rate=0.0)

call(inputs, **kwargs)

Calls the model on new inputs and returns the outputs as tensors.

In this case call() just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Note: This method should not be called directly. It is only meant to be overridden when subclassing tf.keras.Model. To call a model on an input, always use the __call__() method, i.e. model(inputs), which relies on the underlying call() method.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors. training: Boolean or boolean scalar tensor, indicating whether to

run the Network in training mode or inference mode.

mask: A mask or list of masks. A mask can be either a boolean tensor

or None (no mask). For more details, check the guide [here](https://www.tensorflow.org/guide/keras/masking_and_padding).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.