Normalizing Flows by PyTorch

PyTorch implementations of the networks for normalizing flows.

Models

Currently, following networks are implemented.

• Planar flow
  • Rezende and Mohamed 2015, "Variational Inference with Normalizing Flows," [arXiv]
• RealNVP
  • Dinh et al., 2016, "Density Estimation using Real NVP," [arXiv]
• Glow
  • Kingma and Dhariwal 2018, "Glow: Generative Flow with Invertible 1x1 Convolutions," [arXiv] [code]
• Flow++
  • Ho et al., 2019, "Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design," [arXiv] [code]
• MAF
  • Papamakarios et al., 2017, “Masked Autoregressive Flow for Density Estimation,” [arXiv]
• Residual Flow
  • Behrmann et al., 2018, "Residual Flows for Invertible Generative Modeling," [arXiv] [code]
• FFJORD
  • Grathwohl et al., 2018, "FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models," [arXiv] [code]

Note: This repository is for easier understanding of the above networks. Therefore, you should use official source cods if provided.

Setup

Anaconda

By Anaconda, you can easily setup the environment using environment.yml.

$ conda env create -f environment.yml

Pip

If you use pip or other tools, see the dependencies in environment.yml

Run

This repo uses hydra to manage hyper parameters in training and evaluation. See configs folder to check the parameters for each network.

$ python main.py \
    network=[planar, realnvp, glow, flow++, maf, resflow, ffjord]\
    run.distrib=[circles, moons, normals, swiss, s_curve, mnist, cifar10]

Note: Currently, I tested the networks only for 2D density transformation. So, results for 3D densities (swiss and s_curve) and images (mnist and cifar10) could be what you expect.

Results

See results/README.md for more results.

Real NVP

Target	Reproduced	Training

Copyright

MIT License (c) 2020, Tatsuya Yatagawa