Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Last update: Dec 30, 2022

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

cd datasets
wget https://yt-dl.org/downloads/latest/youtube-dl -O youtube-dl
chmod a+rx youtube-dl
python load_videos.py --workers=8
cd ..

Pretrained Headpose Estimator

300W-LP, alpha 1, robust to image quality

Put hopenet_robust_alpha1.pkl here

Train

python train.py --batch_size=4 --gpu_ids=0,1,2,3 --num_epochs=100 (--ckp=10)

On 2080Ti, setting batch_size=4 makes up gpu memory

Evaluate

Reconstruction：

python evaluate.py --ckp=99 --source=r --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

The first frame is used as source by default

Motion transfer：

python evaluate.py --ckp=99 --source=test.png --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

Example after training for 7 days on 4 2080Ti:

Face Frontalization：

python evaluate.py --ckp=99 --source=f --driving=datasets/vox/train/id10192#S5yV10aCP7A#003200#003334.mp4

Acknowlegement

Thanks to NV, Imaginaire, AliaksandrSiarohin and DeepHeadPose

Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

Pretrained Headpose Estimator

Train

Evaluate

Acknowlegement

Owner

worstcoder

Parameterising Simulated Annealing for the Travelling Salesman Problem

Implementation of ViViT: A Video Vision Transformer

Implementation of the method described in the Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

My implementation of transformers related papers for computer vision in pytorch

PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Parallel Latent Tree-Induction for Faster Sequence Encoding

🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

🐾 Semantic segmentation of paws from cute pet images (PyTorch)

The 1st Place Solution of the Facebook AI Image Similarity Challenge (ISC21) : Descriptor Track.

JUSTICE: A Benchmark Dataset for Supreme Court’s Judgment Prediction

Here is the diagnostic tool for BMVC 2021 paper Diagnosing Errors in Video Relation Detectors.

A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulation

Open-L2O: A Comprehensive and Reproducible Benchmark for Learning to Optimize Algorithms

Deep Learning pipeline for motor-imagery classification.

This is a computer vision based implementation of the popular childhood game 'Hand Cricket/Odd or Even' in python

Fast Scattering Transform with CuPy/PyTorch

Offline Multi-Agent Reinforcement Learning Implementations: Solving Overcooked Game with Data-Driven Method

4th place solution to datafactory challenge by Intermarché.

SWA Object Detection

PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"