normflows: A PyTorch Package for Normalizing Flows

2023

Article

ei

Normalizing flows model probability distributions through an expressive tractable density (D. Rezende & Mohamed, 2015; Esteban G. Tabak & Turner, 2013; Esteban G. Tabak & Vanden-Eijnden, 2010). They transform a simple base distribution, such as a Gaussian, through a sequence of invertible functions, which are referred to as layers. These layers typically use neural networks to become very expressive. Flows are ubiquitous in machine learning and have been applied to image generation (Grcić et al., 2021; Kingma & Dhariwal, 2018), text modeling (Wang & Wang, 2019), variational inference (D. Rezende & Mohamed, 2015), approximating Boltzmann distributions (Noé et al., 2019), and many other problems (Kobyzev et al., 2021; Papamakarios et al., 2021). Here, we present normflows, a Python package for normalizing flows. It allows to build normalizing flow models from a suite of base distributions, flow layers, and neural networks. The package is implemented in the popular deep learning framework PyTorch (Paszke et al., 2019), which simplifies the integration of flows in larger machine learning models or pipelines. It supports most of the common normalizing flow architectures, such as Real NVP (Dinh et al., 2017), Glow (Kingma & Dhariwal, 2018), Masked Autoregressive Flows (Papamakarios et al., 2017), Neural Spline Flows (Durkan et al., 2019; Müller et al., 2019), Residual Flows (Chen et al., 2019), and many more. The package can be easily installed via pip and the code is publicly available on GitHub.

@article{normflows2023,
  title = {normflows: A PyTorch Package for Normalizing Flows},
  author = {Stimper, Vincent and Liu, David and Campbell, Andrew and Berenz, Vincent and Ryll, Lukas and Sch{\"o}lkopf, Bernhard and Hernández-Lobato, José Miguel},
  journal = {Journal of Open Source Software},
  volume = {8},
  number = {86},
  pages = {5361},
  publisher = {The Journal of Open Source Software},
  year = {2023},
  doi = {10.21105/joss.05361},
  url = {https://doi.org/10.21105/joss.05361}
}