This is Unofficial Repo. Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (CVPR 2021)

Overview

Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection

This is a PyTorch implementation of the LipForensics paper.

This is an Unofficially implemented codes with some Official code. I made this repo to use more conveniently.

If you want to see the Original code, You can cite this link

You should try the preprocessing, which steps are firstly getting landmarks and then cropping mouth.

Setup

Install packages

pip install -r requirements.txt

Note: we used Python version 3.8 to test this code.

Prepare data

  1. Follow the links below to download the datasets (you will be asked to fill out some forms before downloading):

  2. Extract the frames (e.g. using code in the FaceForensics++ repo.) The filenames of the frames should be as follows: 0000.png, 0001.png, ....

  3. Detect the faces and compute 68 face landmarks. For example, you can use RetinaFace and FAN for good results.

  4. Place face frames and corresponding landmarks into the appropriate directories:

    • For FaceForensics++, FaceShifter, and DeeperForensics, frames for a given video should be placed in data/datasets/Forensics/{dataset_name}/{compression}/images/{video}, where dataset_name is RealFF (real frames from FF++), Deepfakes, FaceSwap, Face2Face, NeuralTextures, FaceShifter, or DeeperForensics. dataset_name is c0, c23, or c40, corresponding to no compression, low compression, and high compression, respectively. video is the video name and should be numbered as follows: 000, 001, .... For example, the frame 0102 of real video 067 at c23 compression is found in data/datasets/Forensics/RealFF/c23/images/067/0102.png
    • For CelebDF-v2, frames for a given video should be placed in data/datasets/CelebDF/{dataset_name}/images/{video} where dataset_name is RealCelebDF, which should include all real videos from the test set, or FakeCelebDF, which should include all fake videos from the test set.
    • For DFDC, frames for a given video should be placed in data/datasets/DFDC/images (both real and fake). The video names from the test set we used in our experiments are given in data/datasets/DFDC/dfdc_all_vids.txt.

    The corresponding computed landmarks for each frame should be placed in .npy format in the directories defined by replacing images with landmarks above (e.g. for video "000", the .npy files for each frame should be placed in data/datasets/Forensics/RealFF/c23/landmarks/000).

  5. To crop the mouth region from each frame for all datasets, run

    python preprocessing/crop_mouths.py --dataset all

    This will write the mouth images into the corresponding cropped_mouths directory.

Evaluate

  • Cross-dataset generalisation (Table 2 in paper):
    1. Download the pretrained model and place into models/weights. This model has been trained on FaceForensics++ (Deepfakes, FaceSwap, Face2Face, and NeuralTextures) and is the one used to get the LipForensics video-level AUC results in Table 2 of the paper, reproduced below:

      CelebDF-v2 DFDC FaceShifter DeeperForensics
      82.4% 73.5% 97.1% 97.6%
    2. To evaluate on e.g. FaceShifter, run

      python evaluate.py --dataset FaceShifter --weights_forgery ./models/weights/lipforensics_ff.pth

Citation

If you find this repo useful for your research, please consider citing the following:

@inproceedings{haliassos2021lips,
  title={Lips Don't Lie: A Generalisable and Robust Approach To Face Forgery Detection},
  author={Haliassos, Alexandros and Vougioukas, Konstantinos and Petridis, Stavros and Pantic, Maja},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={5039--5049},
  year={2021}
}
Owner
Minha Kim
@DASH-Lab on Sungkyunkwan University in Korea
Minha Kim
PyTorch implementation of convolutional neural networks-based text-to-speech synthesis models

Deepvoice3_pytorch PyTorch implementation of convolutional networks-based text-to-speech synthesis models: arXiv:1710.07654: Deep Voice 3: Scaling Tex

Ryuichi Yamamoto 1.8k Jan 08, 2023
Implementation supporting the ICCV 2017 paper "GANs for Biological Image Synthesis"

GANs for Biological Image Synthesis This codes implements the ICCV-2017 paper "GANs for Biological Image Synthesis". The paper and its supplementary m

Anton Osokin 95 Nov 25, 2022
Language-Driven Semantic Segmentation

Language-driven Semantic Segmentation (LSeg) The repo contains official PyTorch Implementation of paper Language-driven Semantic Segmentation. Authors

Intelligent Systems Lab Org 416 Jan 03, 2023
MXNet implementation for: Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution

Octave Convolution MXNet implementation for: Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution Imag

Meta Research 549 Dec 28, 2022
Photographic Image Synthesis with Cascaded Refinement Networks - Pytorch Implementation

Photographic Image Synthesis with Cascaded Refinement Networks-Pytorch (https://arxiv.org/abs/1707.09405) This is a Pytorch implementation of cascaded

Soumya Tripathy 63 Mar 27, 2022
Official Implementation of DDOD (Disentangle your Dense Object Detector), ACM MM2021

Disentangle Your Dense Object Detector This repo contains the supported code and configuration files to reproduce object detection results of Disentan

loveSnowBest 51 Jan 07, 2023
ByteTrack超详细教程!训练自己的数据集&&摄像头实时检测跟踪

ByteTrack超详细教程!训练自己的数据集&&摄像头实时检测跟踪

Double-zh 45 Dec 19, 2022
Official repository for MixFaceNets: Extremely Efficient Face Recognition Networks

MixFaceNets This is the official repository of the paper: MixFaceNets: Extremely Efficient Face Recognition Networks. (Accepted in IJCB2021) https://i

Fadi Boutros 51 Dec 13, 2022
Code for the paper "How Attentive are Graph Attention Networks?"

How Attentive are Graph Attention Networks? This repository is the official implementation of How Attentive are Graph Attention Networks?. The PyTorch

175 Dec 29, 2022
NAS Benchmark in "Prioritized Architecture Sampling with Monto-Carlo Tree Search", CVPR2021

NAS-Bench-Macro This repository includes the benchmark and code for NAS-Bench-Macro in paper "Prioritized Architecture Sampling with Monto-Carlo Tree

35 Jan 03, 2023
OntoProtein: Protein Pretraining With Ontology Embedding

OntoProtein This is the implement of the paper "OntoProtein: Protein Pretraining With Ontology Embedding". OntoProtein is an effective method that mak

ZJUNLP 80 Dec 14, 2022
Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

🍐 quince Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding 🍐 Installation $ git clone

Andrew Jesson 19 Jun 23, 2022
Music Generation using Neural Networks Streamlit App

Music_Gen_Streamlit "Music Generation using Neural Networks" Streamlit App TO DO: Make a run_app.sh Introduction [~5 min] (Sohaib) Team Member names/i

Muhammad Sohaib Arshid 6 Aug 09, 2022
ScriptProfilerPy - Module to visualize where your python script is slow

ScriptProfiler helps you track where your code is slow It provides: Code lines t

Lucas BLP 3 Jun 02, 2022
This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

TSForecasting This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the tim

Rakshitha Godahewa 80 Dec 30, 2022
sktime companion package for deep learning based on TensorFlow

NOTE: sktime-dl is currently being updated to work correctly with sktime 0.6, and wwill be fully relaunched over the summer. The plan is Refactor and

sktime 573 Jan 05, 2023
Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (CVPR 2018).

Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (CVPR2018) By Zilong Huang, Xinggang Wang, Jiasi Wang, Wenyu Liu and J

Zilong Huang 245 Dec 13, 2022
Hand tracking demo for DIY Smart Glasses with a remote computer doing the work

CameraStream This is a demonstration that streams the image from smartglasses to a pc, does the hand recognition on the remote pc and streams the proc

Teemu Laurila 20 Oct 13, 2022
CO-PILOT: COllaborative Planning and reInforcement Learning On sub-Task curriculum

CO-PILOT CO-PILOT: COllaborative Planning and reInforcement Learning On sub-Task curriculum, NeurIPS 2021, Shuang Ao, Tianyi Zhou, Guodong Long, Qingh

Shuang Ao 1 Feb 18, 2022
Denoising Diffusion Probabilistic Models

Denoising Diffusion Probabilistic Models This repo contains code for DDPM training. Based on Denoising Diffusion Probabilistic Models, Improved Denois

Alexander Markov 7 Dec 15, 2022