Official implementation of Densely connected normalizing flows

Last update: Dec 12, 2022

Related tags

Overview

Densely connected normalizing flows

This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster available here.

Setup

CUDA 11.1
Python 3.8

pip install -r requirements.txt
pip install -e .

Training

cd ./experiments/image

CIFAR-10:

python train.py --epochs 400 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.9975 --warmup 5000  --eval_every 1 --check_every 10 --dataset cifar10 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/cifar10_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 420

ImageNet32:

python train.py --epochs 20 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet32 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet32_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 22

ImageNet64:

python train.py --epochs 10 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet64 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet64_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 11

CelebA:

python train.py --epochs 50 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset celeba --augmentation horizontal_flip --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/celeba_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 55

Note: Download instructions for ImageNet and CelebA are defined in denseflow/data/datasets/image/{dataset}.py

Evaluation

CIFAR-10:

python eval_loglik.py --model PATH_TO_MODEL --k 1000 --kbs 50

ImageNet32:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 50

ImageNet64 and CelebA:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 25

Model weights

Model weights are stored here.

Samples generation

Generated samples are stored in PATH_TO_MODEL/samples

python eval_sample.py --model PATH_TO_MODEL

Note: PATH_TO_MODEL has to contain check directory.

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

Significant part of this code benefited from SurVAE [1] code implementation, available under MIT license.

References

[1] Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows. InAdvances in Neural Information Processing Systems 33. Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.

Official implementation of Densely connected normalizing flows

Related tags

Overview

Densely connected normalizing flows

Setup

Training

Evaluation

Model weights

Samples generation

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

References

Owner

Matej Grcić

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks (MAPDN)

Flexible time series feature extraction & processing

An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

DETReg: Unsupervised Pretraining with Region Priors for Object Detection

Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models Benchmark and Efficient Evaluation

Official implementation of "Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform", ICCV 2021

BisQue is a web-based platform designed to provide researchers with organizational and quantitative analysis tools for 5D image data. Users can extend BisQue by implementing containerized ML workflows.

This repository focus on Image Captioning & Video Captioning & Seq-to-Seq Learning & NLP

UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

Model of an AI powered sign language interpreter.

Crowd-sourced Annotation of Human Motion.

Keep CALM and Improve Visual Feature Attribution

Implementation of the paper ''Implicit Feature Refinement for Instance Segmentation''.

Hypernetwork-Ensemble Learning of Segmentation Probability for Medical Image Segmentation with Ambiguous Labels

Fine-Tune EleutherAI GPT-Neo to Generate Netflix Movie Descriptions in Only 47 Lines of Code Using Hugginface And DeepSpeed

Keyword spotting on Arm Cortex-M Microcontrollers

Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks

Portfolio asset allocation strategies: from Markowitz to RNNs

auto-tuning momentum SGD optimizer