Code accompanying the paper "How Tight Can PAC-Bayes be in the Small Data Regime?"

Related tags

Deep Learningpac-bayes-tightness-small-data
Overview

How Tight Can PAC-Bayes be in the Small Data Regime?

This is the code to reproduce all experiments for the following paper:

@inproceedings{Foong:2021:How_Tight_Can_PAC-Bayes_Be,
    title = {How Tight Can {PAC}-{Bayes} Be in the Small Data Regime?},
    year = {2021},
    author = {Andrew Y. K. Foong and Wessel P. Bruinsma and David R. Burt and Richard E. Turner},
    booktitle = {Advances in Neural Information Processing Systems},
    volume = {35},
    eprint = {https://arxiv.org/abs/2106.03542},
}

Every experiment creates a folder in _experiments. The names of the files in those folders should be self-explanatory.

Installation

First, create and activate a virtual environment for Python 3.8.

virtualenv venv -p python3.8 
source venv/bin/activate

Then install an appropriate GPU-accelerated version of PyTorch.

Finally, install the requirements for the project.

pip install -e .

You should now be able to run the below commands.

Generating Datasets

In order to generate the synthetic 1D datasets used, run these commands from inside classification_1d:

python gen_data.py --class_scheme balanced --num_context 30 --name 30-context --num_train_batches 5000 --num_test_batches 64
python gen_data.py --class_scheme balanced --num_context 60 --name 60-context --num_train_batches 5000 --num_test_batches 64

The generated datasets will be in pacbayes/_data_caches

Theory Experiments

See Figure 2 in Section 3 and Appendix G.

python theory_experiments.py --setting det1-1
python theory_experiments.py --setting det1-2
python theory_experiments.py --setting det2-1
python theory_experiments.py --setting det2-1

python theory_experiments.py --setting stoch1
python theory_experiments.py --setting stoch2
python theory_experiments.py --setting stoch3

python theory_experiments.py --setting random --random-seed 1 --random-better-bound maurer
python theory_experiments.py --setting random --random-seed 6 --random-better-bound catoni

GNP Classification Experiments

See Figure 3 and 4 in Section 4 and Appendices I and J. The numbers from the graphs can be found in eval_metrics_no_post_opt.txt (without post optimisation) eval_metrics_post_opt.txt (with post optimisation).

MODEL_NONDDP=maurer MODEL_DDP=maurer-ddp NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer MODEL_DDP=maurer-ddp NUM_CONTEXT=30 ./run_GNP_prop_68.sh
MODEL_NONDDP=catoni MODEL_DDP=catoni-ddp NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=catoni MODEL_DDP=catoni-ddp NUM_CONTEXT=30 ./run_GNP_prop_68.sh
MODEL_NONDDP=convex-nonseparable MODEL_DDP=convex-nonseparable-ddp NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=convex-nonseparable MODEL_DDP=convex-nonseparable-ddp NUM_CONTEXT=30 ./run_GNP_prop_68.sh
MODEL_NONDDP=kl-val MODEL_DDP=kl-val NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=kl-val MODEL_DDP=kl-val NUM_CONTEXT=30 ./run_GNP_prop_68.sh
MODEL_NONDDP=maurer-optimistic MODEL_DDP=maurer-optimistic-ddp NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer-optimistic MODEL_DDP=maurer-optimistic-ddp NUM_CONTEXT=30 ./run_GNP_prop_68.sh
MODEL_NONDDP=maurer-inv MODEL_DDP=maurer-inv-ddp NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer-inv MODEL_DDP=maurer-inv-ddp NUM_CONTEXT=30 ./run_GNP_prop_68.sh
MODEL_NONDDP=maurer-inv-optimistic MODEL_DDP=maurer-inv-optimistic-ddp NUM_CONTEXT=30 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer-inv-optimistic MODEL_DDP=maurer-inv-optimistic-ddp NUM_CONTEXT=30 ./run_GNP_prop_68.sh

MODEL_NONDDP=maurer MODEL_DDP=maurer-ddp NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer MODEL_DDP=maurer-ddp NUM_CONTEXT=60 ./run_GNP_prop_68.sh
MODEL_NONDDP=catoni MODEL_DDP=catoni-ddp NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=catoni MODEL_DDP=catoni-ddp NUM_CONTEXT=60 ./run_GNP_prop_68.sh
MODEL_NONDDP=convex-nonseparable MODEL_DDP=convex-nonseparable-ddp NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=convex-nonseparable MODEL_DDP=convex-nonseparable-ddp NUM_CONTEXT=60 ./run_GNP_prop_68.sh
MODEL_NONDDP=kl-val MODEL_DDP=kl-val NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=kl-val MODEL_DDP=kl-val NUM_CONTEXT=60 ./run_GNP_prop_68.sh
MODEL_NONDDP=maurer-optimistic MODEL_DDP=maurer-optimistic-ddp NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer-optimistic MODEL_DDP=maurer-optimistic-ddp NUM_CONTEXT=60 ./run_GNP_prop_68.sh
MODEL_NONDDP=maurer-inv MODEL_DDP=maurer-inv-ddp NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer-inv MODEL_DDP=maurer-inv-ddp NUM_CONTEXT=60 ./run_GNP_prop_68.sh
MODEL_NONDDP=maurer-inv-optimistic MODEL_DDP=maurer-inv-optimistic-ddp NUM_CONTEXT=60 ./run_GNP_prop_024.sh
MODEL_NONDDP=maurer-inv-optimistic MODEL_DDP=maurer-inv-optimistic-ddp NUM_CONTEXT=60 ./run_GNP_prop_68.sh

MLP Classification Experiments

See Appendix J. The numbers from the graphs can be found in eval_metrics_no_post_opt.txt (without post optimisation) eval_metrics_post_opt.txt (with post optimisation).

MODEL_NONDDP=catoni MODEL_DDP=catoni-ddp NUM_CONTEXT=30 ./run_MLP.sh
MODEL_NONDDP=kl-val MODEL_DDP=kl-val NUM_CONTEXT=30 ./run_MLP.sh

MODEL_NONDDP=catoni MODEL_DDP=catoni-ddp NUM_CONTEXT=60 ./run_MLP.sh
MODEL_NONDDP=kl-val MODEL_DDP=kl-val NUM_CONTEXT=60 ./run_MLP.sh
Owner
GitHub Repository
Source code of NeurIPS 2021 Paper ''Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration''

CaGCN This repo is for source code of NeurIPS 2021 paper "Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration". Paper L

6 Dec 19, 2022
Procedural 3D data generation pipeline for architecture

Synthetic Dataset Generator Authors: Stanislava Fedorova Alberto Tono Meher Shashwat Nigam Jiayao Zhang Amirhossein Ahmadnia Cecilia bolognesi Dominik

Computational Design Institute 49 Nov 25, 2022
The official code of Anisotropic Stroke Control for Multiple Artists Style Transfer

ASMA-GAN Anisotropic Stroke Control for Multiple Artists Style Transfer Proceedings of the 28th ACM International Conference on Multimedia The officia

Six_God 146 Nov 21, 2022
ROS Basics and TurtleSim

Waypoint Follower Anna Garverick This package draws given waypoints, then waits for a service call with a start position to send the turtle to each wa

Anna Garverick 1 Dec 13, 2021
ICRA 2021 "Towards Precise and Efficient Image Guided Depth Completion"

PENet: Precise and Efficient Depth Completion This repo is the PyTorch implementation of our paper to appear in ICRA2021 on "Towards Precise and Effic

232 Dec 25, 2022
Dimension Reduced Turbulent Flow Data From Deep Vector Quantizers

Dimension Reduced Turbulent Flow Data From Deep Vector Quantizers This is an implementation of A Physics-Informed Vector Quantized Autoencoder for Dat

DreamSoul 3 Sep 12, 2022
Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

Xin Wang 69 Oct 13, 2022
MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

MVP Benchmark: Multi-View Partial Point Clouds for Completion and Registration [NEWS] 2021-07-12 [NEW 🎉 ] The submission on Codalab starts! 2021-07-1

PL 93 Dec 21, 2022
Official Implementation for Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation We present a generic image-to-image translation framework, pixel2style2pixel (pSp

2.8k Dec 30, 2022
Listing arxiv - Personalized list of today's articles from ArXiv

Personalized list of today's articles from ArXiv Print and/or send to your gmail

Lilianne Nakazono 5 Jun 17, 2022
Distributed Arcface Training in Pytorch

Distributed Arcface Training in Pytorch

3 Nov 23, 2021
The authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations This is the authors' implementation of Unsupervised Adversarial Learning of

Dwango Media Village 140 Dec 07, 2022
Implementation of the paper ''Implicit Feature Refinement for Instance Segmentation''.

Implicit Feature Refinement for Instance Segmentation This repository is an official implementation of the ACM Multimedia 2021 paper Implicit Feature

Lufan Ma 17 Dec 28, 2022
Image process framework based on plugin like imagej, it is esay to glue with scipy.ndimage, scikit-image, opencv, simpleitk, mayavi...and any libraries based on numpy

Introduction ImagePy is an open source image processing framework written in Python. Its UI interface, image data structure and table data structure a

ImagePy 1.2k Dec 29, 2022
It is a simple library to speed up CLIP inference up to 3x (K80 GPU)

CLIP-ONNX It is a simple library to speed up CLIP inference up to 3x (K80 GPU) Usage Install clip-onnx module and requirements first. Use this trick !

Gerasimov Maxim 93 Dec 20, 2022
Raindrop strategy for Irregular time series

Graph-Guided Network For Irregularly Sampled Multivariate Time Series Overview This repository contains processed datasets and implementation code for

Zitnik Lab @ Harvard 74 Jan 03, 2023
Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals

LapDepth-release This repository is a Pytorch implementation of the paper "Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals" M

Minsoo Song 205 Dec 30, 2022
A facial recognition doorbell system using a Raspberry Pi

Facial Recognition Doorbell This project expands on the person-detecting doorbell system to allow it to identify faces, and announce names accordingly

rydercalmdown 22 Apr 15, 2022
3.8% and 18.3% on CIFAR-10 and CIFAR-100

Wide Residual Networks This code was used for experiments with Wide Residual Networks (BMVC 2016) http://arxiv.org/abs/1605.07146 by Sergey Zagoruyko

Sergey Zagoruyko 1.2k Dec 29, 2022
IEGAN — Official PyTorch Implementation Independent Encoder for Deep Hierarchical Unsupervised Image-to-Image Translation

IEGAN — Official PyTorch Implementation Independent Encoder for Deep Hierarchical Unsupervised Image-to-Image Translation Independent Encoder for Deep

30 Nov 05, 2022