Implementation for the IJCAI2021 work "Beyond the Spectrum: Detecting Deepfakes via Re-synthesis"

Overview

Beyond the Spectrum

Implementation for the IJCAI2021 work "Beyond the Spectrum: Detecting Deepfakes via Re-synthesis" by Yang He, Ning Yu, Margret Keuper and Mario Fritz.

Pretrained Models

We release the model trained on CelebA-HQ dataset with image resolution 1024x1024. For the super resolution, we use 25,000 real images from CelebA-HQ to train it. For the detectors, we use 25,000 real images and 25,000 fake images to train a binary classifier based on ResNet-50.

We release some models as examples to show how to apply our models based on pixel-level or stage5-level reconstruction errors to detect deepfakes. Download link: https://drive.google.com/file/d/1FeIgABjBpjtnXT-Hl6p5a5lpZxINzXwv/view?usp=sharing.

If you have further questions regarding the trained models, please feel free to contact.

Train

  1. Train the super resolution model.

We use Residual Dense Network (RDN) in our work. The following script shows the hyperparameters used in our experiments. To be noticed, we only use 4 images to show how to run the script. For simplicity, you can download the pretrained model from the above link.

bash script/train_super_resolution_celeba.sh [GPU_ID]
  1. Train the detectors.

After obtaining the super resolution, we use pixel-level or stage5-level L1 based recontruction error to train a classifier. The following scripts use 10 training example to show how to train a classifier with a given super resolution model. You may need to adjust the learning rate and number of training epochs in your case.

bash script/train_pixel_pggan.sh [GPU_ID]
  1. Finetune with auxiliary tasks

In order to improve the robustness of our detectors, we introduce auxiliary tasks (i.e., colorization or denoising) into the super resolution model training and finetune the whole model end-to-end. The following scripts show how to train a model with those tasks.

bash script/train_pixel_pggan_colorization.sh [GPU_ID]
bash script/train_stage5_stylegan_denoising.sh [GPU_ID]

Test

Please download our models. You can use pixel-level or stage5-level to perform deepfakes detection.

bash script/test_pixel_celeba.sh [GPU_ID]
bash script/test_stage5_celeba.sh [GPU_ID]

Citation

If our work is useful for you, please cite our paper:

@inproceedings{yang_ijcai21,
  title={Beyond the Spectrum: Detecting Deepfakes via Re-synthesis},
  author={Yang He and Ning Yu and Margret Keuper and Mario Fritz},
  booktitle={30th International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2021}
}

Contact: Yang He ([email protected])

Last update: 08-22-2021

Owner
Yang He
Applied Scientist in Amazon Last Mile PostDoc in CISPA Helmholtz Center for Information Security / PhD in Max Planck Institute for Informatics
Yang He
Local Attention - Flax module for Jax

Local Attention - Flax Autoregressive Local Attention - Flax module for Jax Install $ pip install local-attention-flax Usage from jax import random fr

Phil Wang 16 Jun 16, 2022
HyperPose is a library for building high-performance custom pose estimation applications.

HyperPose is a library for building high-performance custom pose estimation applications.

TensorLayer Community 1.2k Jan 04, 2023
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 04, 2020
N-Person-Check-Checker-Splitter - A calculator app use to divide checks

N-Person-Check-Checker-Splitter This is my from-scratch programmed calculator ap

2 Feb 15, 2022
Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time

Semi Hand-Object Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time (CVPR 2021).

96 Dec 27, 2022
Hand Gesture Volume Control is AIML based project which uses image processing to control the volume of your Computer.

Hand Gesture Volume Control Modules There are basically three modules Handtracking Program Handtracking Module Volume Control Program Handtracking Pro

VITTAL 1 Jan 12, 2022
Use graph-based analysis to re-classify stocks and to improve Markowitz portfolio optimization

Dynamic Stock Industrial Classification Use graph-based analysis to re-classify stocks and experiment different re-classification methodologies to imp

Sheng Yang 10 Dec 05, 2022
This is the code for the paper "Contrastive Clustering" (AAAI 2021)

Contrastive Clustering (CC) This is the code for the paper "Contrastive Clustering" (AAAI 2021) Dependency python=3.7 pytorch=1.6.0 torchvision=0.8

Yunfan Li 210 Dec 30, 2022
[AI6122] Text Data Management & Processing

[AI6122] Text Data Management & Processing is an elective course of MSAI, SCSE, NTU, Singapore. The repository corresponds to the AI6122 of Semester 1, AY2021-2022, starting from 08/2021. The instruc

HT. Li 1 Jan 17, 2022
official code for dynamic convolution decomposition

Revisiting Dynamic Convolution via Matrix Decomposition (ICLR 2021) A pytorch implementation of DCD. If you use this code in your research please cons

Yunsheng Li 110 Nov 23, 2022
GraphLily: A Graph Linear Algebra Overlay on HBM-Equipped FPGAs

GraphLily: A Graph Linear Algebra Overlay on HBM-Equipped FPGAs GraphLily is the first FPGA overlay for graph processing. GraphLily supports a rich se

Cornell Zhang Research Group 39 Dec 13, 2022
Bayesian Inference Tools in Python

BayesPy Bayesian Inference Tools in Python Our goal is, given the discrete outcomes of events, estimate the distribution of categories. Using gradient

Max Sklar 99 Dec 14, 2022
Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

Rafael Berral Soler 71 Jan 05, 2023
Speedy Implementation of Instance-based Learning (IBL) agents in Python

A Python library to create single or multi Instance-based Learning (IBL) agents that are built based on Instance Based Learning Theory (IBLT) 1 Instal

0 Nov 18, 2021
Official code for MPG2: Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN

This is the official code for Multi-attribute Pizza Generator (MPG2): Cross-domain Attribute Control with Conditional StyleGAN. Paper Demo Setup Envir

Fangda Han 5 Sep 01, 2022
Compositional Sketch Search

Compositional Sketch Search Official repository for ICIP 2021 Paper: Compositional Sketch Search Requirements Install and activate conda environment c

Alexander Black 8 Sep 06, 2021
DAN: Unfolding the Alternating Optimization for Blind Super Resolution

DAN-Basd-on-Openmmlab DAN: Unfolding the Alternating Optimization for Blind Super Resolution We reproduce DAN via mmediting based on open-sourced code

AlexZou 72 Dec 13, 2022
Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs

Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs MATLAB implementation of the paper: P. Mercado, F. Tudisco, and M. Hein,

Pedro Mercado 6 May 26, 2022
Model Zoo of BDD100K Dataset

Model Zoo of BDD100K Dataset

ETH VIS Group 200 Dec 27, 2022
ICLR2021 (Under Review)

Self-Supervised Time Series Representation Learning by Inter-Intra Relational Reasoning This repository contains the official PyTorch implementation o

Haoyi Fan 58 Dec 30, 2022