Official code for "On the Frequency Bias of Generative Models", NeurIPS 2021

Overview

Frequency Bias of Generative Models

Generator Testbed Discriminator Testbed

This repository contains official code for the paper On the Frequency Bias of Generative Models.

You can find detailed usage instructions for analyzing standard GAN-architectures and your own models below.

If you find our code or paper useful, please consider citing

@inproceedings{Schwarz2021NEURIPS,
  title = {On the Frequency Bias of Generative Models},
  author = {Schwarz, Katja and Liao, Yiyi and Geiger, Andreas},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year = {2021}
}

Installation

Please note, that this repo requires one GPU for running. First you have to make sure that you have all dependencies in place. The simplest way to do so, is to use anaconda.

You can create an anaconda environment called fbias using

conda env create -f environment.yml
conda activate fbias

Generator Testbed

You can run a demo of our generator testbed via:

chmod +x ./scripts/demo_generator_testbed.sh
./scripts/demo_generator_testbed.sh

This will train the Generator of Progressive Growing GAN to regress a single image. Further, the training progression on the image regression, spectrum, and spectrum error are summarized in output/generator_testbed/baboon64/pggan/eval.

In general, to analyze the spectral properties of a generator architecture you can train a model by running

python generator_testbed.py *EXPERIMENT_NAME* *PATH/TO/CONFIG*

This script should create a folder output/generator_testbed/*EXPERIMENT_NAME* where you can find the training progress. To evaluate the spectral properties of the trained model run

python eval_generator.py *EXPERIMENT_NAME* --psnr --image-evolution --spectrum-evolution --spectrum-error-evolution

This will print the average PSNR of the regressed images and visualize image evolution, spectrum evolution, and spectrum error evolution in output/generator_testbed/*EXPERIMENT_NAME*/eval.

Discriminator Testbed

You can run a demo of our discriminator testbed via:

chmod +x ./scripts/demo_discriminator_testbed.sh
./scripts/demo_discriminator_testbed.sh

This will train the Discriminator of Progressive Growing GAN to regress a single image. Further, the training progression on the image regression, spectrum, and spectrum error are summarized in output/discriminator_testbed/baboon64/pggan/eval.

In general, to analyze the spectral properties of a discriminator architecture you can train a model by running

python discriminator_testbed.py *EXPERIMENT_NAME* *PATH/TO/CONFIG*

This script should create a folder output/discriminator_testbed/*EXPERIMENT_NAME* where you can find the training progress. To evaluate the spectral properties of the trained model run

python eval_discriminator.py *EXPERIMENT_NAME* --psnr --image-evolution --spectrum-evolution --spectrum-error-evolution

This will print the average PSNR of the regressed images and visualize image evolution, spectrum evolution, and spectrum error evolution in output/discriminator_testbed/*EXPERIMENT_NAME*/eval.

Datasets

Toyset

You can generate a toy dataset with Gaussian peaks as spectrum by running

cd data
python toyset.py 64 100
cd ..

This creates a folder data/toyset/ and generates 100 images of resolution 64x64 pixels.

CelebA-HQ

Download celebA_hq. Then, update data:root: *PATH/TO/CELEBA_HQ* in the config file.

Other datasets

The config setting data:root: *PATH/TO/DATA* needs to point to a folder with the training images. You can use any dataset which follows the folder structure

*PATH/TO/DATA*/xxx.png
*PATH/TO/DATA*/xxy.png
...

By default, the images are center-cropped and optionally resized to the resolution specified in the config file underdata:resolution. Note, that you can also use a subset of images via data:subset.

Architectures

StyleGAN Support

In addition to Progressive Growing GAN, this repository supports analyzing the following architectures

For this, you need to initialize the stylegan3 submodule by running

git pull --recurse-submodules
cd models/stylegan3/stylegan3
git submodule init
git submodule update
cd ../../../

Next, you need to install any additional requirements for this repo. You can do this by running

conda activate fbias
conda env update --file environment_sg3.yml --prune

You can now analyze the spectral properties of the StyleGAN architectures by running

# StyleGAN2
python generator_testbed.py baboon64/StyleGAN2 configs/generator_testbed/sg2.yaml
python discriminator_testbed.py baboon64/StyleGAN2 configs/discriminator_testbed/sg2.yaml
# StyleGAN3
python generator_testbed.py baboon64/StyleGAN3 configs/generator_testbed/sg3.yaml

Other architectures

To analyze any other network architectures, you can add the respective model file (or submodule) under models. You then need to write a wrapper class to integrate the architecture seamlessly into this code base. Examples for wrapper classes are given in

  • models/stylegan2_generator.py for the Generator
  • models/stylegan2_discriminator.py for the Discriminator

Further Information

This repository builds on Lars Mescheder's awesome framework for GAN training. Further, we utilize code from the Stylegan3-repo and GenForce.

Code for CPM-2 Pre-Train

CPM-2 Pre-Train Pre-train CPM-2 此分支为110亿非 MoE 模型的预训练代码,MoE 模型的预训练代码请切换到 moe 分支 CPM-2技术报告请参考link。 0 模型下载 请在智源资源下载页面进行申请,文件介绍如下: 文件名 描述 参数大小 100000.tar

Tsinghua AI 136 Dec 28, 2022
Building a real-time environment using webcam frame division in OpenCV and classify cropped images using a fine-tuned vision transformers on hybryd datasets samples for facial emotion recognition.

Visual Transformer for Facial Emotion Recognition (FER) This project has the aim to build an efficient Visual Transformer for the Facial Emotion Recog

Mario Sessa 8 Dec 12, 2022
Efficient Two-Step Networks for Temporal Action Segmentation (Neurocomputing 2021)

Efficient Two-Step Networks for Temporal Action Segmentation This repository provides a PyTorch implementation of the paper Efficient Two-Step Network

8 Apr 16, 2022
Fastshap: A fast, approximate shap kernel

fastshap: A fast, approximate shap kernel fastshap was designed to be: Fast Calculating shap values can take an extremely long time. fastshap utilizes

Samuel Wilson 22 Sep 24, 2022
git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]

FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (CVPR 2021) This repo contains the implementation of our state-of-the-art fewshot ob

233 Dec 29, 2022
Explaining Hyperparameter Optimization via PDPs

Explaining Hyperparameter Optimization via PDPs This repository gives access to an implementation of the methods presented in the paper submission “Ex

2 Nov 16, 2022
Pytorch Implementation of Interaction Networks for Learning about Objects, Relations and Physics

Interaction-Network-Pytorch Pytorch Implementraion of Interaction Networks for Learning about Objects, Relations and Physics. Interaction Network is a

117 Nov 05, 2022
WormMovementSimulation - 3D Simulation of Worm Body Movement with Neurons attached to its body

Generate 3D Locomotion Data This module is intended to create 2D video trajector

1 Aug 09, 2022
The final project of "Applying AI to 2D Medical Imaging Data" of "AI for Healthcare" nanodegree - Udacity.

Pneumonia Detection from X-Rays Project Overview In this project, you will apply the skills that you have acquired in this 2D medical imaging course t

Omar Laham 1 Jan 14, 2022
Data and code for the paper "Importance of Kernel Bandwidth in Quantum Machine Learning"

Reproducibility materials for "Importance of Kernel Bandwidth in Quantum Machine Learning" Repo structure: code contains Python scripts used to genera

Ruslan Shaydulin 3 Oct 23, 2022
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity

[ICLR 2022] Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity by Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elen

VITA 18 Dec 31, 2022
The Turing Change Point Detection Benchmark: An Extensive Benchmark Evaluation of Change Point Detection Algorithms on real-world data

Turing Change Point Detection Benchmark Welcome to the repository for the Turing Change Point Detection Benchmark, a benchmark evaluation of change po

The Alan Turing Institute 85 Dec 28, 2022
Pytorch implementation for "Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter".

Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter This is a pytorch-based implementation for paper Implicit Feature Alignme

wangtianwei 61 Nov 12, 2022
A new video text spotting framework with Transformer

TransVTSpotter: End-to-end Video Text Spotter with Transformer Introduction A Multilingual, Open World Video Text Dataset and End-to-end Video Text Sp

weijiawu 67 Jan 03, 2023
El-Gamal on Elliptic Curve (Python)

El-Gamal-on-EC El-Gamal on Elliptic Curve (Python) References: https://docsdrive.com/pdfs/ansinet/itj/2005/299-306.pdf https://arxiv.org/ftp/arxiv/pap

3 May 04, 2022
Credit fraud detection in Python using a Jupyter Notebook

Credit-Fraud-Detection - Credit fraud detection in Python using a Jupyter Notebook , using three classification models (Random Forest, Gaussian Naive Bayes, Logistic Regression) from the sklearn libr

Ali Akram 4 Dec 28, 2021
SOLO and SOLOv2 for instance segmentation, ECCV 2020 & NeurIPS 2020.

SOLO: Segmenting Objects by Locations This project hosts the code for implementing the SOLO algorithms for instance segmentation. SOLO: Segmenting Obj

Xinlong Wang 1.5k Dec 31, 2022
ZeroVL - The official implementation of ZeroVL

This repository contains source code necessary to reproduce the results presente

31 Nov 04, 2022
Summary of related papers on visual attention

This repo is built for paper: Attention Mechanisms in Computer Vision: A Survey paper Vision-Attention-Papers Channel attention Spatial attention Temp

MenghaoGuo 2.1k Dec 30, 2022
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations

Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Code repo for paper Trans-Encoder: Unsupervised sentence-pa

Amazon 101 Dec 29, 2022