Differentiable Annealed Importance Sampling (DAIS)

This repository contains the code to reproduce the DAIS results from the paper Differentiable Annealed Importance Sampling and the Perils of Gradient Noise.

Bayesian Linear Regression

Please first install JAX (https://github.com/google/jax), then reproduce our results by running the following:

python blr/dais.py --gamma 0.0
python blr/dais.py --gamma 0.9
python blr.dais.py --bsize 100

Variational Autoencoder

Please first install PyTorch (https://pytorch.org/).

For training, one example command for DAIS with $K = 10$ and $S = 5$ (to adapt annealing scheme, add --adapt_beta):

python vae/mnist_train.py --lf_step 10 --n_particles 5 --lf_lrate 0.08

After training, you can find the saved checkpoint and evaluate it (e.g., using AIS):

python vae/mnist_eval.py --ais --n_particles 10 --lf_step 10000 --lf_lrate 0.05 --resume xxx

Also, you can reproduce the evaulation results of AIS/HAIS/DAIS by runing vae/mnist_eval_scaling.py:

python vae/mnist_eval_scaling.py --linear_beta --lf_lrate 0.08 --n_particles 10 --resume xxx

Citation

To cite this work, please use

@inproceedings{
	zhang2021differentiable,
	title={Differentiable Annealed Importance Sampling and the Perils of Gradient Noise},
	author={Guodong Zhang and Kyle Hsu and Jianing Li and Chelsea Finn and Roger Baker Grosse},
	booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
	year={2021},
	url={https://openreview.net/forum?id=6rqjgrL7Lq}
}

Differentiable Annealed Importance Sampling (DAIS)

Related tags

Overview

Differentiable Annealed Importance Sampling (DAIS)

Bayesian Linear Regression

Variational Autoencoder

Citation

Owner

Guodong Zhang

Improving 3D Object Detection with Channel-wise Transformer

Evolving neural network parameters in JAX.

Pytorch implementation of ProjectedGAN

Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch

A Graph Neural Network Tool for Recovering Dense Sub-graphs in Random Dense Graphs.

Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling @ INTERSPEECH 2021 Accepted

Official Pytorch implementation of "Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes", CVPR 2022

The code for SAG-DTA: Prediction of Drug–Target Affinity Using Self-Attention Graph Network.

Cockpit is a visual and statistical debugger specifically designed for deep learning.

[CVPR 2021] MiVOS - Mask Propagation module. Reproduced STM (and better) with training code :star2:. Semi-supervised video object segmentation evaluation.

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Logistic Bandit experiments. Official code for the paper "Jointly Efficient and Optimal Algorithms for Logistic Bandits".

[NeurIPS2021] Code Release of K-Net: Towards Unified Image Segmentation

Prompts - Read a textfile of prompts and import into anki via ankiconnect

This is project is the implementation of the DeepShift: Towards Multiplication-Less Neural Networks paper

The VeriNet toolkit for verification of neural networks

ZEBRA: Zero Evidence Biometric Recognition Assessment

Code accompanying the paper "How Tight Can PAC-Bayes be in the Small Data Regime?"

A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK