Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A collection of automation aids to connect various database systems into Lookout for Metrics

A collection of automation aids to connect various database systems into Lookout for Metrics

AWS Samples 3 Apr 28, 2022
arweave-nft-uploader is a Python tool to improve the experience of uploading NFTs to the Arweave storage for use with the Metaplex Candy Machine.

arweave-nft-uploader arweave-nft-uploader is a Python tool to improve the experience of uploading NFTs to the Arweave storage for use with the Metaple

0xEnrico 84 Dec 26, 2022
⭐️ Pyro String Generator ⭐️ Genrate String Session Using this bot.Made by TeamUltronX 🔥

⭐️ Pyro String Generator ⭐️ Genrate String Session Using this bot.Made by TeamUltronX 🔥 Configs: API_HASH Get from Here. API_ID Get from Here. API_KE

TheUltronX 2 Dec 16, 2022
Discord Account Generator that will create Account with hCaptcha bypass. Using socks4 proxies

Account-Generator [!] This was made for education. Please use socks4 proxies for nice experiences. [!] Please install these modules - "pip3 install ht

RyanzSantos 10 Feb 23, 2022
pymobiledevice fork with more recent coding standards and many more features

Description Features Installation Usage Sending your own messages Lockdown messages Instruments messages Example Lockdown services com.apple.instrumen

255 Dec 28, 2022
Wats2PDF - Convert whatsapp exported chat(without media) into a readable pdf format

Wats2PDF convert whatsApp exported chat into a readable pdf format. convert with

5 Apr 26, 2022
Wanna play on the Overwatch NA servers?

OverwatchRegionSwapper-NA- Wanna play on the Overwatch NA servers? Use at you own risk. LIST OF IPs EU Netherlands: From: 5.42.168.0 to: 5.42.175.255

1 Jun 08, 2022
DragDev Maintained Instance Of discord.py

discord.py - DragDev Flavour A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. The Future of discord.py

DragDev Studios 3 Aug 27, 2022
💬 Send iMessages using Python through the Shortcuts app.

py-imessage-shortcuts Send iMessages using Python through the Shortcuts app. Requires macOS Monterey (macOS 12) or later. Compatible with Apple Silico

Kevin Schaich 10 Nov 30, 2022
Weather_besac is a French twitter bot that tweet the weather of the city of Besançon in Franche-Comté in France every day at 8am and 4pm.

Weather Bot Besac Weather_besac is a French twitter bot that tweet the weather of the city of Besançon in Franche-Comté in France every day at 8am and

Rgld_ 1 Nov 15, 2021
Detects members having unicode names. Public bot: @scarletwitchprobot

✨ Scarletwitch bot ✨ Detects unicode names members in a tg chat & provides a option to take action on that user ! Public bot: @scarletwitchprobot Supp

ÁÑÑÍHÌLÅTØR SPÄRK 18 Nov 12, 2022
Modern, privacy-friendly, and detailed web analytics that works without cookies or JS.

Modern, privacy-friendly, and cookie-free web analytics. Getting started » Screenshots • Features • Office Hours Motivation There are a lot of web ana

R. Miles McCain 2.1k Jan 03, 2023
A Discord webhook spammer made in Python.

A Python made Discord webhook spammer usually used for token loggers to spam them/delete them original by cattyn I only made it so u can change the avatar to whatever u want instead of it being hardc

notperry1234567890 15 Dec 15, 2021
PlexAutoSkip - Automatically skip content in Plex

PlexAutoSkip Automatically skip tagged content in Plex A background python scrip

Michael Higgins 97 Dec 21, 2022
Telegram music & video bot direct play music

⚡ NOINOI MUSIC PLAYER 🎵 SUPERFAST MUSIC BOT WHO CAN DIRECT PLAY SONG ON TELEGRAM VOICE CHAT ALSO CAN PLAY VIDEO ON VOICE CHATS ✨ Heroku Deploy YOU CA

noinoi-X 1 Dec 28, 2021
Using DST's API with Python

A short guide on how to access Denmark's Statistics API with python, together with a helper class that facilitates the collection of data and metadata from any DST's table

Alessandro Martinello 16 Dec 02, 2022
vk.com API python wrapper

Python vk.com API wrapper This is a vk.com (the largest Russian social network) python API wrapper. The goal is to support all API methods (current an

Dmitry Voronin 371 Dec 29, 2022
Anti Spam/NSFW Telegram Bot Written In Python With Pyrogram.

✨ SpamProtectionRobot ✨ Anti Spam/NSFW Telegram Bot Written In Python With Pyrogram. Requirements Python = 3.7 Install Locally Or On A VPS $ git clon

Akshay Rajput 46 Dec 13, 2022
A really easy way to display your spotify listening status on spotify.

Spotify playing README A really easy way to display your spotify listening status on READMEs and Websites too! Demo Here's the embed from the site. Cu

Sunrit Jana 21 Nov 06, 2022
Ethereum transactions and wallet information for people you follow on Twitter.

ethFollowing Ethereum transactions and wallet information for people you follow on Twitter. Set up Setup python environment (requires python 3.8): vir

Brian Donohue 2 Dec 28, 2021