Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Quot-a-lecture - Lecture transcript question extraction

Setup virtualenv venv source venv/bin/activate pip install -r requirements.txt

Pratyaksh Sharma 5 Sep 12, 2022
LOL-banner - A discord bot that bans anybody playing league of legends

LOL-banner A discord bot that bans anybody playing league of legends This bot ha

bsd_witch 46 Dec 17, 2022
A Telegram bot to upload files from Telegram or Direct links to Google Drive.

Google Drive Uploader Telegram Bot A Telegram bot to upload files from Telegram or Direct links to Google Drive. Features Telegram files support. Dire

IDNCoderX 21 Dec 05, 2022
Touca SDK for Python

Touca SDK For Python Touca helps you understand the true impact of your day to day code changes on the behavior and performance of your overall softwa

Touca 12 May 18, 2022
Create CDK projects with projen

The Projenator: I'll be back! Description This is a CDKv2 project that takes the grind out of setting up new cdk projects/implementations by using aut

Andrew 2 Dec 11, 2021
A project that alerts me when there's a dog outside so I can go look at it.

Dog Detector A project that alerts me when there's a dog outside so I can go look at it. Tech Specs This script uses the YOLOv3 object detection model

rydercalmdown 58 Jul 29, 2022
domhttpx is a google search engine dorker with HTTP toolkit built with python, can make it easier for you to find many URLs/IPs at once with fast time.

domhttpx is a google search engine dorker with HTTP toolkit built with python, can make it easier for you to find many URLs/IPs at once with fast time

Naufal Ardhani 59 Dec 04, 2022
Monitor your Binance portfolio

Binance Report Bot The intent of this bot is to take a snapshot of your binance wallet, e.g. the current balances and store it for further plotting. I

37 Oct 29, 2022
A Simple Telegram Maths Calculator Bot

Calculator-Bot-v1 A Simple Telegram Maths Calculator Bot Demo BOT LINK: Variables Variables Required Variables API_HASH: Get

ᗪᗩᖇK ✞Oᖇᗪ 1 Dec 18, 2021
Bancos de Dados Relacionais (SQL) na AWS com Amazon RDS.

Bancos de Dados Relacionais (SQL) na AWS com Amazon RDS Explorando o Amazon RDS, um serviço de provisionamente e gerenciamento de banco de dados relac

Lucas Magalhães 1 Dec 05, 2021
Running Performance Calculator

Running Performance Calculator 👉 Have you ever wondered if you ran 10km at 2000

Davide Liu 6 Oct 26, 2022
A powerfull Telegram Leech Bot

owner of this repo :- Abijthkutty contact me :- Abijth Telegram Torrent and Direct links Leecher Dont Abuse The Repo ... this is intented to run in Sm

αвιנтн 9 Jun 11, 2022
The easiest way to deploy this Bot

How To Host The easiest way to deploy this Bot Update Channe

Isekai Reszz 1 Jan 23, 2022
Python interface to the World Bank Indicators and Climate APIs

wbpy A Python interface to the World Bank Indicators and Climate APIs. Readthedocs Github source World Bank API docs The Indicators API lets you acces

Matt Duck 47 Oct 31, 2022
A script to automatically update bot status at GitHub as well as in Telegram channel.

Support BotStatus ~ A simple & short repository to show your bot's status in your GitHub README.md file as well as in you channel. ⚠️ This repo should

Jainam Oswal 55 Dec 13, 2022
A Simple Telegram Bot By @AsmSafone to Download Files From Mega.nz and Upload It to Telegram

MegaDL-Bot A Simple Telegram Bot By @AsmSafone to Download Files From Mega.nz and Upload It to Telegram Features No Login Required All Mega.nz File Li

SAF ONE 92 Dec 02, 2022
An API wrapper around the pythonanywhere's API.

pyaww An API wrapper around the pythonanywhere's API. The name stands for pythonanywherewrapper. 100% api coverage most of the codebase is documented

7 Dec 11, 2022
DB-Drive-CSV - This is app is can be used to access CSV file as JSON from Google Drive.

DB Drive CSV This is app is can be used to access CSV file as JSON from Google Drive. How To Use Create file/ upload file to Google Drive There's 2 fi

Hartawan Bahari M. 5 Oct 20, 2022
A modified Sequential and NLP based Bot

A modified Sequential and NLP based Bot I improvised this bot a bit with some implementations as a part of my own hobby project :) Note: I do not own

Jay Desale 2 Jan 07, 2022
🦊 Powerfull Discord Nitro Generator

🦊 Follow me here 🦊 Discord | YouTube | Github ☕ Usage 💻 Downloading git clone https://github.com/KanekiWeb/Nitro-Generator/new/main pip insta

Kaneki 104 Jan 02, 2023