Fake videos detection by tracing the source using video hashing retrieval.

Last update: Dec 22, 2022

Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

Introduction
Train or Test
Tracing
- Trace Samples
Localization
- Localization Samples

✨ Introduction

VTL

Video Tracing and Tampering Localization (VTL). A novel framework to detect fake video (clipping, cropping, blur, etc.) by tracing the source video of fake video. 1) Training hash centers as HCs. 2) Finding index of source video from HCs. 3) Masking the different between fake video and source video as a result of comparison (auxiliary information).

Trace Samples and Acc of HashBits

Although the source videos are very similar, we can accurately find the source videos of the fake video clips.

DFTL Dataset Samples

Same person with different scenes. You can download full 16 minutes videos of source video and fake video by follows link.

Source Videos

Fake Videos

Different fake videos from same source.

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

The first gif of boat is source video, and remaining five videos generated by different inpainting methods.

🔬 Train or Test

Datasets Download

BaiduNetdisk code：VTLs

actors: Source videos and fake videos of full 16 minutes. You can use these videos to make richer datasets.
DFTL: Dataset of DFTL, the DFTL build from actors.
DAVIS2016-TL: Extension of DAVIS2016

Extract to the same directory as the code (vtl). Example:

├─other files
├─project
│  ├─vrf: dataset of DFTL
│  ├─inpainting: dataset of DAVIS2016-TL
│  └─vtl: our code
│      ├─CSQ: Central Similarity Quantization for Efficient Image and Video Retrieval
│      ├─dmac: Compared method of Localization
│      └─codes

Train

Pretrained models and hash centers

pip install -r requirements.txt

Model	DFTL	DAVIS2016-TL
ViTHash	64-1024bits	64-1024bits
Generator	link	link

Parameters

local_rank: gpu id
path: dataset path
type: choice dataloader
- 0: DFTL dataloader, dir name is vrf
- 1: DAVIS2016-TL dataloader, dir name is inpainting

Train ViTHash

python train_h.py --local_rank=0 --path=../vrf --type=0 --bits=128

Train Generator

python train_g.py --local_rank=0 --path=../vrf --type=0

Test

Test IOU

The test script will test Generator of VTL and DMAC together on DFTL and DAVIS2016-TL. You can modify it for yourself.

python test_iou.py

Test ViTHash

type: choice dataloader
- 0: DFTL dataloader, dir name is vrf
- 1: DAVIS2016-TL dataloader, dir name is inpainting
path: dataset path
hashbits: 128 256 512 or 1024, will load different pre-trained model and hash JSON file.

python test.py 1 ../inpainting 512

Test CSQ

cd ./CSQ
run test script

python hash_test_vrf.py --dataset=Inpainting --pretrained_3d=./Inpainting_64bits.pth

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...

Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Deep Unsupervised Image Hashing by Maximizing Bit Entropy This is the PyTorch implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hash

62 Dec 30, 2022

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

71 Dec 22, 2022

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021) Zeyu Wang, Sherry Qiu, Nicole Feng, Holly Rushmeier, Leonard McMill

23 Dec 9, 2022

Enhancing Knowledge Tracing via Adversarial Training

Enhancing Knowledge Tracing via Adversarial Training This repository contains source code for the paper "Enhancing Knowledge Tracing via Adversarial T

14 Oct 24, 2022

Implementation of light baking system for ray tracing based on Activision's UberBake

Vulkan Light Bakary MSU Graphics Group Student's Diploma Project Treefonov Andrey [GitHub] [LinkedIn] Project Goal The goal of the project is to imple

7 Dec 27, 2022

Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

TensorGeodesic Ray tracing of a Schwarzschild black hole written entirely in TensorFlow. Dependencies: Python 3 TensorFlow 2.x numpy matplotlib About

5 Jan 15, 2022

Space-event-trace - Tracing service for spaceteam events

space-event-trace Tracing service for TU Wien Spaceteam events. This service is

2 Jan 4, 2022

Activity image-based video retrieval

Cross-modal-retrieval Our approach is focus on Activity Image-to-Video Retrieval (AIVR) task. The compared methods are state-of-the-art single modalit

75 Oct 21, 2021

A Joint Video and Image Encoder for End-to-End Retrieval

Frozen️ in Time ❄️ ️️️️ ⏳ A Joint Video and Image Encoder for End-to-End Retrieval project page | arXiv | webvid-data Repository containing the code,

225 Dec 25, 2022

Comments

the pre-trained model

I downloaded the provided pre-trained model and the test acc in DFTL and DAVIS2016-TL reached 100%, is this correct? Why is it so much higher than in the article?

opened by ymhzyj 1

Fake videos detection by tracing the source using video hashing retrieval.

Related tags

Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

✨ Introduction

VTL

Trace Samples and Acc of HashBits

DFTL Dataset Samples

Source Videos

Fake Videos

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

🔬 Train or Test

Datasets Download

Train

Test

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...

Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Enhancing Knowledge Tracing via Adversarial Training

Implementation of light baking system for ray tracing based on Activision's UberBake

Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

Space-event-trace - Tracing service for spaceteam events

Activity image-based video retrieval

A Joint Video and Image Encoder for End-to-End Retrieval

Comments

the pre-trained model

Releases(v1.0)

Owner

Predictive AI layer for existing databases.

How will electric vehicles affect traffic congestion and energy consumption: an integrated modelling approach

Implementation of a protein autoregressive language model, but with autoregressive infilling objective (editing subsequences capability)

Binary classification for arrythmia detection with ECG datasets.

AttGAN: Facial Attribute Editing by Only Changing What You Want (IEEE TIP 2019)

Manifold Alignment for Semantically Aligned Style Transfer

[CVPR 2021] Generative Hierarchical Features from Synthesizing Images

BLEND: A Fast, Memory-Efficient, and Accurate Mechanism to Find Fuzzy Seed Matches

Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

Official code for On Path Integration of Grid Cells: Group Representation and Isotropic Scaling (NeurIPS 2021)

code for our BMVC 2021 paper "HCV: Hierarchy-Consistency Verification for Incremental Implicitly-Refined Classification"

Adaptive Dropblock Enhanced GenerativeAdversarial Networks for Hyperspectral Image Classification

Multi-Anchor Active Domain Adaptation for Semantic Segmentation (ICCV 2021 Oral)

reimpliment of DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation

Code repository for paper `Skeleton Merger: an Unsupervised Aligned Keypoint Detector`.

VOGUE: Try-On by StyleGAN Interpolation Optimization

Realtime_Multi-Person_Pose_Estimation

Code for the SIGGRAPH 2022 paper "DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds."

Python implementation of MULTIseq barcode alignment using fuzzy string matching and GMM barcode assignment

Dynamic Realtime Animation Control