Eff video representation - Efficient video representation through neural fields

Last update: Jan 06, 2023

Related tags

Deep Learning eff_video_representation

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

Download MPI sintel dataset from here

2. GMA optical flow estimator

To obtain optical flow estimations for pretraining, we are using GMA from here. Note that it dose not have to do with our identity.

3. Training

Training neural residual flow fields (NRFF)

# frame 0 - 6
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 0 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start0_jq98_hf96
# frame 7 - 13
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 7 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start7_jq98_hf96
# frame 14 - 20
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 14 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start14_jq98_hf96
# frame 21 - 27
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 21 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start21_jq98_hf96

Training baseline (SIREN)

python train_video.py --data-dir {sintel dataset training directory} --video-name alley_1 --hidden-features 256 --num-frames 28 --lr 0.001 --training-step 30000 --tag baseline_siren_hf256

4. Examples

alley_2.mp4

HoneyBee.mp4

Eff video representation - Efficient video representation through neural fields

Related tags

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

2. GMA optical flow estimator

3. Training

4. Examples

Owner

A rough implementation of the paper "A Steering Algorithm for Redirected Walking Using Reinforcement Learning"

✂️ EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video.

Generic template to bootstrap your PyTorch project with PyTorch Lightning, Hydra, W&B, and DVC.

Progressive Coordinate Transforms for Monocular 3D Object Detection

DeepLearning Anomalies Detection with Bluetooth Sensor Data

A simple editor for captions in .SRT file extension

Kaggleship: Kaggle Notebooks

Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Code for the paper: Fighting Fake News: Image Splice Detection via Learned Self-Consistency

BankNote-Net: Open dataset and encoder model for assistive currency recognition

the code for our CVPR 2021 paper Bilateral Grid Learning for Stereo Matching Network [BGNet]

Implementation of the paper ''Implicit Feature Refinement for Instance Segmentation''.

Python scripts for performing lane detection using the LSTR model in ONNX

A Closer Look at Structured Pruning for Neural Network Compression

Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks

TextWorld is a sandbox learning environment for the training and evaluation of reinforcement learning (RL) agents on text-based games.

Video Background Music Generation with Controllable Music Transformer (ACM MM 2021 Oral)

GNNAdvisor: An Efficient Runtime System for GNN Acceleration on GPUs

Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches

Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition"

Eff video representation - Efficient video representation through neural fields

Related tags

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

2. GMA optical flow estimator

3. Training

4. Examples

Owner

A rough implementation of the paper "A Steering Algorithm for Redirected Walking Using Reinforcement Learning"

✂️ EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video.

Generic template to bootstrap your PyTorch project with PyTorch Lightning, Hydra, W&B, and DVC.

Progressive Coordinate Transforms for Monocular 3D Object Detection

DeepLearning Anomalies Detection with Bluetooth Sensor Data

A simple editor for captions in .SRT file extension

Kaggleship: Kaggle Notebooks

Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Code for the paper: Fighting Fake News: Image Splice Detection via Learned Self-Consistency

BankNote-Net: Open dataset and encoder model for assistive currency recognition

the code for our CVPR 2021 paper Bilateral Grid Learning for Stereo Matching Network [BGNet]

Implementation of the paper ''Implicit Feature Refinement for Instance Segmentation''.

Python scripts for performing lane detection using the LSTR model in ONNX

A Closer Look at Structured Pruning for Neural Network Compression

Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks

​TextWorld is a sandbox learning environment for the training and evaluation of reinforcement learning (RL) agents on text-based games.

Video Background Music Generation with Controllable Music Transformer (ACM MM 2021 Oral)

GNNAdvisor: An Efficient Runtime System for GNN Acceleration on GPUs

Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches

Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition"

TextWorld is a sandbox learning environment for the training and evaluation of reinforcement learning (RL) agents on text-based games.