Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

Overview

FFD Source Code

Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

The proposed network framework with attention mechanism

Project Webpage

See the MSU CVLab website for project details and access to the DFFD dataset.

http://cvlab.cse.msu.edu/project-ffd.html

Notes

This code is provided as example code, and may not reflect a specific combination of hyper-parameters presented in the paper.

Description of contents

  • xception.py: Defines the Xception network with the attention mechanism
  • train*.py: Train the model on the train data
  • test*.py: Evaluate the model on the test data

Acknowledgements

If you use or refer to this source code, please cite the following paper:

@inproceedings{cvpr2020-dang,
  title={On the Detection of Digital Face Manipulation},
  author={Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, Anil Jain},
  booktitle={In Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2020)},
  address={Seattle, WA},
  year={2020}
}
Comments
  • Is it possible to release the script for generating edited images by FaceApp?

    Is it possible to release the script for generating edited images by FaceApp?

    Hi, Thanks for releasing the code and dataset! Part of your dataset is generated by FaceApp (using automated scripts running on android devices). I am wondering if you could also release this android script? I also plan to generate some edited images using FaceApp, and an automated script will be quite helpful!! Thanks!

    opened by zjxgithub 2
  • Question about mask images in dataset

    Question about mask images in dataset

    Thank you for releasing the code and the DFFD dataset!

    I noticed that in the "faceapp" part of the dataset, there is a ground-truth manipulation masks image for each fake image. How are these mask images generated?

    The paper mentioned that the ground-truth manipulation mask were calculated by source images and fake images, but I still did not understand how.

    Thank you for answering my question. :)

    opened by piddnad 2
  • Serveral question about dataset

    Serveral question about dataset

    Thanks for releasing the code and the dataset. I have some questions for the dataset,

    • In align_faces/align_faces.m inside scripts.zip, there is a file called box.txt. But I can't find it anywhere. It seems crucial to align and crop the images.

    image

    • All of the images in dataset are in the resolution of 299x299. I wonder how did you process the images in CelebA. I remember the aligned and cropped image in CelebA are in the resolution of 128x128.
    opened by wheatdog 2
  • attention map and gt mask matching

    attention map and gt mask matching

    Hi, thanks for your work. I have a small question. The attention map size is 19x19, but the gt mask (diff image) is 299x299. Are they matched by downsampling gt mask?

    opened by neverUseThisName 1
  • Are label information leaked in testing process?

    Are label information leaked in testing process?

    Thanks for uploading your code and dataset. After a short view I'm considering your predicting process is like: generating masks with scripts on test data, using test data and their masks to feed into trained model to predict. But I was confused that in your test.py file, you get dataset like this:

    def get_dataset():
      return Dataset('test', BATCH_SIZE, CONFIG['img_size'], CONFIG['map_size'], CONFIG['norms'], SEED)
    

    then you differ masks of real and fake photos by using their labels in dataset.py:

      def __getitem__(self, index):
        im_name = self.images[index]
        img = self.load_image(im_name)
        if self.label_name == 'Real':
          msk = torch.zeros(1,19,19)
        else:
          msk = self.load_mask(im_name.replace('Fake/', 'Mask/'))
        return {'img': img, 'msk': msk, 'lab': self.label, 'im_name': im_name}
    

    Is it fair to distinguish masks by label_name in the testing process? I also wonder how to create Mask/ folder when you predict fake images that donot have corresponding real images?

    If i misunderstand anything please correct me, thanks a lot!

    opened by insomnia1996 0
  • May I know where I can find the imagenet pretrained model?

    May I know where I can find the imagenet pretrained model?

    Hi,

    For using pretrained model: xception-b5690688.pth, may I know where I can find the model specified here: https://github.com/JStehouwer/FFD_CVPR2020/blob/master/xception.py#L243

    Thanks.

    opened by ilovecv 2
  • Error in get_batch in train.py

    Error in get_batch in train.py

    Greetings,

    Many thanks to your wok. I am very interested in your work and I want to try out your model. When I ran the train*.py, I encounter the following issue , here are part of the error messages.

    batch = [next(_.generator, None) for _ in self.datasets]
    

    File "D:\Fake Detector\attention_map_to_detect_manipulation\FFD_CVPR2020\dataset.py", line 91, in self = reduction.pickle.load(from_parent)batch = [next(_.generator, None) for _ in self.datasets]

    File "D:\Fake Detector\attention_map_to_detect_manipulation\FFD_CVPR2020\dataset.py", line 73, in get_batch EOFError: Ran out of input

    and reduction.dump(process_obj, to_child) File "C:\Users\xxx\anaconda3\envs\d2l\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle 'generator' object

    What I did is just make directory data/train/Real(Fake) and place my images dataset into the corresponding folder and then ran the train.py. However, it seems it can't work. May I ask whether I missed anything. I am running the program in windows system and I don't know that will affect as well.

    opened by bitrookie 1
  • Use pretrained model to classify own data?

    Use pretrained model to classify own data?

    Hi @JStehouwer - thank you so much for the awesome code (v2.1)!

    I am trying to use your pretrained model on my own images in order to try out the classifier.

    Are you able to confirm:

    • Filename and format of pretrained model
    • Whether anything else is needed to perform the above classification

    Thanks again

    opened by jtlz2 4
  • dataset questions

    dataset questions

    1、 Whether the published dataset ( FFHQ、FaceAPP、StarGAN、PGGAN、StyleGAN ) has been randomly selected ? And How to generate starGAN mask, how to determine the specific CelebA picture used ? 2、 I have downloaded the FF++、CelebA and DeepFaceLab dataset, how to randomly select the training set, test set and verification set ? And how to set the random seed ? 3、 Which data sets need align processing, and how, please specify ?

    Thank you for your work, it is very good, I will follow your work, but now the problem of dataset makes my work difficult, I hope to get your help.

    opened by miaoct 2
Releases(v2.1)
A multi-entity Transformer for multi-agent spatiotemporal modeling.

baller2vec This is the repository for the paper: Michael A. Alcorn and Anh Nguyen. baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotempor

Michael A. Alcorn 56 Nov 15, 2022
Research code for CVPR 2021 paper "End-to-End Human Pose and Mesh Reconstruction with Transformers"

MeshTransformer ✨ This is our research code of End-to-End Human Pose and Mesh Reconstruction with Transformers. MEsh TRansfOrmer is a simple yet effec

Microsoft 473 Dec 31, 2022
SlotRefine: A Fast Non-Autoregressive Model forJoint Intent Detection and Slot Filling

SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling Reference Main paper to be cited (Di Wu et al., 2020) @article

Moore 34 Nov 03, 2022
A Deep Reinforcement Learning Framework for Stock Market Trading

DQN-Trading This is a framework based on deep reinforcement learning for stock market trading. This project is the implementation code for the two pap

61 Jan 01, 2023
The code for the NSDI'21 paper "BMC: Accelerating Memcached using Safe In-kernel Caching and Pre-stack Processing".

BMC The code for the NSDI'21 paper "BMC: Accelerating Memcached using Safe In-kernel Caching and Pre-stack Processing". BibTex entry available here. B

Orange 383 Dec 16, 2022
GAN-based Matrix Factorization for Recommender Systems

GAN-based Matrix Factorization for Recommender Systems This repository contains the datasets' splits, the source code of the experiments and their res

Ervin Dervishaj 9 Nov 06, 2022
A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks without the use of any outside machine learning libraries - all from scratch.

Kordel K. France 2 Nov 14, 2022
Simulation-based inference for the Galactic Center Excess

Simulation-based inference for the Galactic Center Excess Siddharth Mishra-Sharma and Kyle Cranmer Abstract The nature of the Fermi gamma-ray Galactic

Siddharth Mishra-Sharma 3 Jan 21, 2022
A basic implementation of Layer-wise Relevance Propagation (LRP) in PyTorch.

Layer-wise Relevance Propagation (LRP) in PyTorch Basic unsupervised implementation of Layer-wise Relevance Propagation (Bach et al., Montavon et al.)

Kai Fabi 28 Dec 26, 2022
Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection (NimPme) The official implementation of Novel Instances Mining with

12 Sep 08, 2022
A unofficial pytorch implementation of PAN(PSENet2): Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network

Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network Requirements pytorch 1.1+ torchvision 0.3+ pyclipper opencv3 gcc

zhoujun 400 Dec 26, 2022
Orbivator AI - To Determine which features of data (measurements) are most important for diagnosing breast cancer and find out if breast cancer occurs or not.

Orbivator_AI Breast Cancer Wisconsin (Diagnostic) GOAL To Determine which features of data (measurements) are most important for diagnosing breast can

anurag kumar singh 1 Jan 02, 2022
Train CPPNs as a Generative Model, using Generative Adversarial Networks and Variational Autoencoder techniques to produce high resolution images.

cppn-gan-vae tensorflow Train Compositional Pattern Producing Network as a Generative Model, using Generative Adversarial Networks and Variational Aut

hardmaru 343 Dec 29, 2022
Automated image registration. Registrationimation was too much of a mouthful.

alignimation Automated image registration. Registrationimation was too much of a mouthful. This repo contains the code used for my blog post Alignimat

Ethan Rosenthal 9 Oct 13, 2022
PN-Net a neural field-based framework for depth estimation from single-view RGB images.

PN-Net We present a neural field-based framework for depth estimation from single-view RGB images. Rather than representing a 2D depth map as a single

1 Oct 02, 2021
DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time Introduction This is official implementation for DR-GAN (IEEE TCS

Kang Liao 18 Dec 23, 2022
A Jinja extension (compatible with Flask and other frameworks) to compile and/or compress your assets.

A Jinja extension (compatible with Flask and other frameworks) to compile and/or compress your assets.

Jayson Reis 94 Nov 21, 2022
PerfFuzz: Automatically Generate Pathological Inputs for C/C++ programs

PerfFuzz Performance problems in software can arise unexpectedly when programs are provided with inputs that exhibit pathological behavior. But how ca

Caroline Lemieux 125 Nov 18, 2022
Awesome Monocular 3D detection

Awesome Monocular 3D detection Paper list of 3D detetction, keep updating! Contents Paper List 2022 2021 2020 2019 2018 2017 2016 KITTI Results Paper

Zhikang Zou 184 Jan 04, 2023
Official code for 'Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentationon Complex Urban Driving Scenes'

PEBAL This repo contains the Pytorch implementation of our paper: Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urb

Yu Tian 117 Jan 03, 2023