Implementation for the IJCAI2021 work "Beyond the Spectrum: Detecting Deepfakes via Re-synthesis"

Overview

Beyond the Spectrum

Implementation for the IJCAI2021 work "Beyond the Spectrum: Detecting Deepfakes via Re-synthesis" by Yang He, Ning Yu, Margret Keuper and Mario Fritz.

Pretrained Models

We release the model trained on CelebA-HQ dataset with image resolution 1024x1024. For the super resolution, we use 25,000 real images from CelebA-HQ to train it. For the detectors, we use 25,000 real images and 25,000 fake images to train a binary classifier based on ResNet-50.

We release some models as examples to show how to apply our models based on pixel-level or stage5-level reconstruction errors to detect deepfakes. Download link: https://drive.google.com/file/d/1FeIgABjBpjtnXT-Hl6p5a5lpZxINzXwv/view?usp=sharing.

If you have further questions regarding the trained models, please feel free to contact.

Train

  1. Train the super resolution model.

We use Residual Dense Network (RDN) in our work. The following script shows the hyperparameters used in our experiments. To be noticed, we only use 4 images to show how to run the script. For simplicity, you can download the pretrained model from the above link.

bash script/train_super_resolution_celeba.sh [GPU_ID]
  1. Train the detectors.

After obtaining the super resolution, we use pixel-level or stage5-level L1 based recontruction error to train a classifier. The following scripts use 10 training example to show how to train a classifier with a given super resolution model. You may need to adjust the learning rate and number of training epochs in your case.

bash script/train_pixel_pggan.sh [GPU_ID]
  1. Finetune with auxiliary tasks

In order to improve the robustness of our detectors, we introduce auxiliary tasks (i.e., colorization or denoising) into the super resolution model training and finetune the whole model end-to-end. The following scripts show how to train a model with those tasks.

bash script/train_pixel_pggan_colorization.sh [GPU_ID]
bash script/train_stage5_stylegan_denoising.sh [GPU_ID]

Test

Please download our models. You can use pixel-level or stage5-level to perform deepfakes detection.

bash script/test_pixel_celeba.sh [GPU_ID]
bash script/test_stage5_celeba.sh [GPU_ID]

Citation

If our work is useful for you, please cite our paper:

@inproceedings{yang_ijcai21,
  title={Beyond the Spectrum: Detecting Deepfakes via Re-synthesis},
  author={Yang He and Ning Yu and Margret Keuper and Mario Fritz},
  booktitle={30th International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2021}
}

Contact: Yang He ([email protected])

Last update: 08-22-2021

Owner
Yang He
Applied Scientist in Amazon Last Mile PostDoc in CISPA Helmholtz Center for Information Security / PhD in Max Planck Institute for Informatics
Yang He
Sparse R-CNN: End-to-End Object Detection with Learnable Proposals, CVPR2021

End-to-End Object Detection with Learnable Proposal, CVPR2021

Peize Sun 1.2k Dec 27, 2022
A model which classifies reviews as positive or negative.

SentiMent Analysis In this project I built a model to classify movie reviews fromn the IMDB dataset of 50K reviews. WordtoVec : Neural networks only w

Rishabh Bali 2 Feb 09, 2022
A machine learning malware analysis framework for Android apps.

🕵️ A machine learning malware analysis framework for Android apps. ☢️ DroidDetective is a Python tool for analysing Android applications (APKs) for p

James Stevenson 77 Dec 27, 2022
CRF-RNN for Semantic Image Segmentation - PyTorch version

This repository contains the official PyTorch implementation of the "CRF-RNN" semantic image segmentation method, published in the ICCV 2015

Sadeep Jayasumana 170 Dec 13, 2022
Simple-Image-Classification - Simple Image Classification Code (PyTorch)

Simple-Image-Classification Simple Image Classification Code (PyTorch) Yechan Kim This repository contains: Python3 / Pytorch code for multi-class ima

Yechan Kim 8 Oct 29, 2022
Official repository of the paper Privacy-friendly Synthetic Data for the Development of Face Morphing Attack Detectors

SMDD-Synthetic-Face-Morphing-Attack-Detection-Development-dataset Official repository of the paper Privacy-friendly Synthetic Data for the Development

10 Dec 12, 2022
Data Preparation, Processing, and Visualization for MoVi Data

MoVi-Toolbox Data Preparation, Processing, and Visualization for MoVi Data, https://www.biomotionlab.ca/movi/ MoVi is a large multipurpose dataset of

Saeed Ghorbani 51 Nov 27, 2022
Locally Constrained Self-Attentive Sequential Recommendation

LOCKER This is the pytorch implementation of this paper: Locally Constrained Self-Attentive Sequential Recommendation. Zhankui He, Handong Zhao, Zhe L

Zhankui (Aaron) He 8 Jul 30, 2022
《Train in Germany, Test in The USA: Making 3D Object Detectors Generalize》(CVPR 2020)

Train in Germany, Test in The USA: Making 3D Object Detectors Generalize This paper has been accpeted by Conference on Computer Vision and Pattern Rec

Xiangyu Chen 101 Jan 02, 2023
The codes and related files to reproduce the results for Image Similarity Challenge Track 1.

ISC-Track1-Submission The codes and related files to reproduce the results for Image Similarity Challenge Track 1. Required dependencies To begin with

Wenhao Wang 115 Jan 02, 2023
RP-GAN: Stable GAN Training with Random Projections

RP-GAN: Stable GAN Training with Random Projections This repository contains a reference implementation of the algorithm described in the paper: Behna

Ayan Chakrabarti 20 Sep 18, 2021
A pre-trained model with multi-exit transformer architecture.

ElasticBERT This repository contains finetuning code and checkpoints for ElasticBERT. Towards Efficient NLP: A Standard Evaluation and A Strong Baseli

fastNLP 48 Dec 14, 2022
A simplified framework and utilities for PyTorch

Here is Poutyne. Poutyne is a simplified framework for PyTorch and handles much of the boilerplating code needed to train neural networks. Use Poutyne

GRAAL/GRAIL 534 Dec 17, 2022
A Python Package for Convex Regression and Frontier Estimation

pyStoNED pyStoNED is a Python package that provides functions for estimating multivariate convex regression, convex quantile regression, convex expect

Sheng Dai 17 Jan 08, 2023
Example of semantic segmentation in Keras

keras-semantic-segmentation-example Example of semantic segmentation in Keras Single class example: Generated data: random ellipse with random color o

53 Mar 23, 2022
User-friendly bulk RNAseq deconvolution using simulated annealing

Welcome to cellanneal - The user-friendly application for deconvolving omics data sets. cellanneal is an application for deconvolving biological mixtu

11 Dec 16, 2022
The MLOps platform for innovators 🚀

​ DS2.ai is an integrated AI operation solution that supports all stages from custom AI development to deployment. It is an AI-specialized platform service that collects data, builds a training datas

9 Jan 03, 2023
Repository for scripts and notebooks from the book: Programming PyTorch for Deep Learning

Repository for scripts and notebooks from the book: Programming PyTorch for Deep Learning

Ian Pointer 368 Dec 17, 2022
Pytorch implementation of the paper: "SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Image Deraining"

SAPNet This repository contains the official Pytorch implementation of the paper: "SAPNet: Segmentation-Aware Progressive Network for Perceptual Contr

11 Oct 17, 2022
PyTorch implementation of CVPR'18 - Perturbative Neural Networks

This is an attempt to reproduce results in Perturbative Neural Networks paper. See original repo for details.

Michael Klachko 57 May 14, 2021