Hierarchical Few-Shot Generative Models

Last update: Dec 12, 2022

Overview

Hierarchical Few-Shot Generative Models

Giorgio Giannone, Ole Winther

This repo contains code and experiments for the paper Hierarchical Few-Shot Generative Models.

Website: https://georgosgeorgos.github.io/hierarchical-few-shot-generative-models/

Settings

Clone the repo:

git clone https://github.com/georgosgeorgos/hierarchical-few-shot-generative-models
cd hierarchical-few-shot-generative-models

Create and activate the conda env:

conda env create -f environment.yml
conda activate hfsgm

The code has been tested on Ubuntu 18.04, Python 3.6 and CUDA 11.3

We use wandb for visualization. The first time you run the code you will need to login.

Data

We provide preprocessed Omniglot dataset.

From the main folder, copy the data in data/omniglot_ns/:

wget https://github.com/georgosgeorgos/hierarchical-few-shot-generative-models/releases/download/Omniglot/omni_train_val_test.pkl

For CelebA you need to download the dataset from here.

Dataset

In dataset we provide utilities to process and augment datasets in the few-shot setting. Each dataset is a large collection of small sets. Sets can be created dynamically. The dataset/base.py file collects basic info about the datasets. For binary datasets (omniglot_ns.py) we augment using flipping and rotations. For RGB datasets (celeba.py) we use only flipping.

Experiment

In experiment we implement scripts for model evaluation, experiments and visualizations.

attention.py - visualize attention weights and heads for models with learnable aggregations (LAG).
cardinality.py - compute ELBOs for different input set size: [1, 2, 5, 10, 20].
classifier_mnist.py - few-shot classifiers on MNIST.
kl_layer.py - compute KL over z and c for each layer in latent space.
marginal.py - compute approximate log-marginal likelihood with 1K importance samples.
refine_vis.py - visualize refined samples.
sampling_rgb.py - reconstruction, conditional, refined, unconditional sampling for RGB datasets.
sampling_transfer.py - reconstruction, conditional, refined, unconditional sampling on transfer datasets.
sampling.py - reconstruction, conditional, refined, unconditional sampling for binary datasets.
transfer.py - compute ELBOs on MNIST, DoubleMNIST, TripleMNIST.

Model

In model we implement baselines and model variants.

base.py - base class for all the models.
vae.py - Variational Autoencoder (VAE).
ns.py - Neural Statistician (NS).
tns.py - NS with learnable aggregation (NS-LAG).
cns.py - NS with convolutional latent space (CNS).
ctns.py - CNS with learnable aggregation (CNS-LAG).
hfsgm.py - Hierarchical Few-Shot Generative Model (HFSGM).
thfsgm.py - HFSGM with learnable aggregation (HFSGM-LAG).
chfsgm.py - HFSGM with convolutional latent space (CHFSGM).
cthfsgm.py - CHFSGM with learnable aggregation (CHFSGM-LAG).

Script

Scripts used for training the models in the paper.

To run a CNS on Omniglot:

sh script/main_cns.sh GPU_NUMBER omniglot_ns

Train a model

To train a generic model run:

python main.py --name {VAE, NS, CNS, CTNS, CHFSGM, CTHFSGM} \
               --model {vae, ns, cns, ctns, chfsgm, cthfsgm} \
               --augment \
               --dataset omniglot_ns \
               --likelihood binary \
               --hidden-dim 128 \
               --c-dim 32 \
               --z-dim 32 \
               --output-dir /output \
               --alpha-step 0.98 \
               --alpha 2 \
               --adjust-lr \
               --scheduler plateau \
               --sample-size {2, 5, 10} \
               --sample-size-test {2, 5, 10} \
               --num-classes 1 \
               --learning-rate 1e-4 \
               --epochs 400 \
               --batch-size 100 \
               --tag (optional string)

If you do not want to save logs, use the flag --dry_run. This flag will call utils/trainer_dry.py instead of trainer.py.

Acknowledgments

A lot of code and ideas borrowed from:

You might also like...

Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Hypercorrelation Squeeze for Few-Shot Segmentation This is the implementation of the paper "Hypercorrelation Squeeze for Few-Shot Segmentation" by Juh

165 Dec 28, 2022

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

Cross Transformers - Pytorch (wip) Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch Install $ pip install cross-t

40 Dec 22, 2022

Official repository for Few-shot Image Generation via Cross-domain Correspondence (CVPR '21)

Few-shot Image Generation via Cross-domain Correspondence Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A. Efros, Yong Jae Lee, Eli Shechtman, Richard Zh

251 Dec 11, 2022

[CVPR 2021] Few-shot 3D Point Cloud Semantic Segmentation

Few-shot 3D Point Cloud Semantic Segmentation Created by Na Zhao from National University of Singapore Introduction This repository contains the PyTor

117 Dec 27, 2022

Few-Shot Graph Learning for Molecular Property Prediction

Few-shot Graph Learning for Molecular Property Prediction Introduction This is the source code and dataset for the following paper: Few-shot Graph Lea

94 Dec 12, 2022

Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs

Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs This is an implemetation of the paper Few-shot Relation Extraction via Baye

36 Nov 22, 2022

The implementation of PEMP in paper "Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes"

Prior-Enhanced network with Meta-Prototypes (PEMP) This is the PyTorch implementation of PEMP. Overview of PEMP Meta-Prototypes & Adaptive Prototypes

8 Oct 14, 2021

Code and data of the ACL 2021 paper: Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision

MetaAdaptRank This repository provides the implementation of meta-learning to reweight synthetic weak supervision data described in the paper Few-Shot

5 Jun 16, 2022

Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (CVPR 2021)

ASGNet The code is for the paper "Adaptive Prototype Learning and Allocation for Few-Shot Segmentation" (accepted to CVPR 2021) [arxiv] Overview data/

91 Dec 23, 2022

Hierarchical Few-Shot Generative Models

Related tags

Overview

Hierarchical Few-Shot Generative Models

Settings

Data

Dataset

Experiment

Model

Script

Train a model

Acknowledgments

You might also like...

Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

Official repository for Few-shot Image Generation via Cross-domain Correspondence (CVPR '21)

[CVPR 2021] Few-shot 3D Point Cloud Semantic Segmentation

Few-Shot Graph Learning for Molecular Property Prediction

Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs

The implementation of PEMP in paper "Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes"

Code and data of the ACL 2021 paper: Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision

Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (CVPR 2021)

Releases(Omniglot)

Omniglot(Jun 9, 2021)

Owner

Giorgio Giannone

A large dataset of 100k Google Satellite and matching Map images, resembling pix2pix's Google Maps dataset.

This repository is related to an Arabic tutorial, within the tutorial we discuss the common data structure and algorithms and their worst and best case for each, then implement the code using Python.

The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We significantly improve the systematic generalization of transformer models on a variety of datasets using simple tricks and careful considerations.

Mercury: easily convert Python notebook to web app and share with others

SwinTrack: A Simple and Strong Baseline for Transformer Tracking

CSPML (crystal structure prediction with machine learning-based element substitution)

Mapping Conditional Distributions for Domain Adaptation Under Generalized Target Shift

Implementation for the EMNLP 2021 paper "Interactive Machine Comprehension with Dynamic Knowledge Graphs".

Continuous Augmented Positional Embeddings (CAPE) implementation for PyTorch

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

PaddleBoBo是基于PaddlePaddle和PaddleSpeech、PaddleGAN等开发套件的虚拟主播快速生成项目

Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

LowRankModels.jl is a julia package for modeling and fitting generalized low rank models.

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Rank 3 : Source code for OPPO 6G Data Generation Challenge

A PyTorch implementation of the continual learning experiments with deep neural networks

ByteTrack(Multi-Object Tracking by Associating Every Detection Box)のPythonでのONNX推論サンプル

Hack Camera, Microphone, Location, Clipboard With Just a Link. Also, Get Many Details About Victim's Device. And So On...

This project is for a Twitter bot that monitors a bird feeder in my backyard. Any detected birds are identified and posted to Twitter.

Galaxy images labelled by morphology (shape). Aimed at ML development and teaching