Fake videos detection by tracing the source using video hashing retrieval.

Last update: Dec 22, 2022

Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

Introduction
Train or Test
Tracing
- Trace Samples
Localization
- Localization Samples

✨ Introduction

VTL

Video Tracing and Tampering Localization (VTL). A novel framework to detect fake video (clipping, cropping, blur, etc.) by tracing the source video of fake video. 1) Training hash centers as HCs. 2) Finding index of source video from HCs. 3) Masking the different between fake video and source video as a result of comparison (auxiliary information).

Trace Samples and Acc of HashBits

Although the source videos are very similar, we can accurately find the source videos of the fake video clips.

DFTL Dataset Samples

Same person with different scenes. You can download full 16 minutes videos of source video and fake video by follows link.

Source Videos

Fake Videos

Different fake videos from same source.

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

The first gif of boat is source video, and remaining five videos generated by different inpainting methods.

🔬 Train or Test

Datasets Download

BaiduNetdisk code：VTLs

actors: Source videos and fake videos of full 16 minutes. You can use these videos to make richer datasets.
DFTL: Dataset of DFTL, the DFTL build from actors.
DAVIS2016-TL: Extension of DAVIS2016

Extract to the same directory as the code (vtl). Example:

├─other files
├─project
│  ├─vrf: dataset of DFTL
│  ├─inpainting: dataset of DAVIS2016-TL
│  └─vtl: our code
│      ├─CSQ: Central Similarity Quantization for Efficient Image and Video Retrieval
│      ├─dmac: Compared method of Localization
│      └─codes

Train

Pretrained models and hash centers

pip install -r requirements.txt

Model	DFTL	DAVIS2016-TL
ViTHash	64-1024bits	64-1024bits
Generator	link	link

Parameters

local_rank: gpu id
path: dataset path
type: choice dataloader
- 0: DFTL dataloader, dir name is vrf
- 1: DAVIS2016-TL dataloader, dir name is inpainting

Train ViTHash

python train_h.py --local_rank=0 --path=../vrf --type=0 --bits=128

Train Generator

python train_g.py --local_rank=0 --path=../vrf --type=0

Test

Test IOU

The test script will test Generator of VTL and DMAC together on DFTL and DAVIS2016-TL. You can modify it for yourself.

python test_iou.py

Test ViTHash

type: choice dataloader
- 0: DFTL dataloader, dir name is vrf
- 1: DAVIS2016-TL dataloader, dir name is inpainting
path: dataset path
hashbits: 128 256 512 or 1024, will load different pre-trained model and hash JSON file.

python test.py 1 ../inpainting 512

Test CSQ

cd ./CSQ
run test script

python hash_test_vrf.py --dataset=Inpainting --pretrained_3d=./Inpainting_64bits.pth

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...

Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Deep Unsupervised Image Hashing by Maximizing Bit Entropy This is the PyTorch implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hash

62 Dec 30, 2022

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

71 Dec 22, 2022

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021) Zeyu Wang, Sherry Qiu, Nicole Feng, Holly Rushmeier, Leonard McMill

23 Dec 9, 2022

Enhancing Knowledge Tracing via Adversarial Training

Enhancing Knowledge Tracing via Adversarial Training This repository contains source code for the paper "Enhancing Knowledge Tracing via Adversarial T

14 Oct 24, 2022

Implementation of light baking system for ray tracing based on Activision's UberBake

Vulkan Light Bakary MSU Graphics Group Student's Diploma Project Treefonov Andrey [GitHub] [LinkedIn] Project Goal The goal of the project is to imple

7 Dec 27, 2022

Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

TensorGeodesic Ray tracing of a Schwarzschild black hole written entirely in TensorFlow. Dependencies: Python 3 TensorFlow 2.x numpy matplotlib About

5 Jan 15, 2022

Space-event-trace - Tracing service for spaceteam events

space-event-trace Tracing service for TU Wien Spaceteam events. This service is

2 Jan 4, 2022

Activity image-based video retrieval

Cross-modal-retrieval Our approach is focus on Activity Image-to-Video Retrieval (AIVR) task. The compared methods are state-of-the-art single modalit

75 Oct 21, 2021

A Joint Video and Image Encoder for End-to-End Retrieval

Frozen️ in Time ❄️ ️️️️ ⏳ A Joint Video and Image Encoder for End-to-End Retrieval project page | arXiv | webvid-data Repository containing the code,

225 Dec 25, 2022

Comments

the pre-trained model

I downloaded the provided pre-trained model and the test acc in DFTL and DAVIS2016-TL reached 100%, is this correct? Why is it so much higher than in the article?

opened by ymhzyj 1

Fake videos detection by tracing the source using video hashing retrieval.

Related tags

Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

✨ Introduction

VTL

Trace Samples and Acc of HashBits

DFTL Dataset Samples

Source Videos

Fake Videos

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

🔬 Train or Test

Datasets Download

Train

Test

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...

Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Enhancing Knowledge Tracing via Adversarial Training

Implementation of light baking system for ray tracing based on Activision's UberBake

Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

Space-event-trace - Tracing service for spaceteam events

Activity image-based video retrieval

A Joint Video and Image Encoder for End-to-End Retrieval

Comments

the pre-trained model

Releases(v1.0)

Owner

💊 A 3D Generative Model for Structure-Based Drug Design (NeurIPS 2021)

PyTorch implementation of Interpretable Explanations of Black Boxes by Meaningful Perturbation

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

xitorch: differentiable scientific computing library

PyTorch implementation of the paper Deep Networks from the Principle of Rate Reduction

Revisting Open World Object Detection

Make differentially private training of transformers easy for everyone

This repository contains the data and code for the paper "Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors" ([email protected])

A disassembler for the RP2040 Programmable I/O State-machine!

Mosaic of Object-centric Images as Scene-centric Images (MosaicOS) for long-tailed object detection and instance segmentation.

Deep Watershed Transform for Instance Segmentation

Code release for BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images

For IBM Quantum Challenge Africa 2021, 9 September (07:00 UTC) - 20 September (23:00 UTC).

Object Tracking and Detection Using OpenCV

TensorFlow, PyTorch and Numpy layers for generating Orthogonal Polynomials

Dataset and Code for ICCV 2021 paper "Real-world Video Super-resolution: A Benchmark Dataset and A Decomposition based Learning Scheme"

Resco: A simple python package that report the effect of deep residual learning

python 93% acc. CNN Dogs Vs Cats ( Pytorch )

Computer Vision application in the web

Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch