Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A Flask & Twilio Secret Santa app.

🎄 ✨ Secret Santa Twilio ✨ 📱 A contactless Secret Santa game built with Python, Flask and Twilio! Prerequisites 📝 A Twilio account. Sign up here ngr

Sangeeta Jadoonanan 5 Dec 23, 2021
Senexia - A powerful telegram bot to manage your groups as effectively as possible

⚡ Kenechi bot ⚡ A Powerful, Smart And Simple Group Manager ... Written with AioG

Akhi 2 Jan 11, 2022
Python-based Snapchat score booster using pyautogui module

Snapchat Snapscore Botter Python-based Snapchat score booster using pyautogui module. Click here to report bugs. Usage Download ZIP here and extract t

477 Dec 31, 2022
Aplicação dos metodos de classificação em 3 diferentes banco de dados. Usando...

Machine Learning - Métodos de classificação Base de Dados utilizadas: Dados de crédito Dados do Census Métodos de classificação aplicados: Naive Bayes

1 Jan 18, 2022
The gPodder podcast client.

___ _ _ ____ __ _| _ \___ __| |__| |___ _ _ |__ / / _` | _/ _ \/ _` / _` / -_) '_| |_ \ \__, |_| \___/\__,_\__,_\___|_| |_

gPodder and related projects 1.1k Jan 04, 2023
🐍 The official Python client library for Google's discovery based APIs.

Google API Client This is the Python client library for Google's discovery based APIs. To get started, please see the docs folder. These client librar

Google APIs 6.2k Jan 08, 2023
Instagram-follower-bot - An Instagram follower bot written in Python

Instagram Follower Bot An Instagram follower bot written in Python. The bot follows the follower of which account you want. e.g. (You want to follow @

Aytaç Kaşoğlu 1 Dec 31, 2021
ANKIT-OS/TG-MUSIC-PLAYER a special repository. Its Is A Telegram Bot To Play To Play Music In Voice Chat

🔥 🎶 TG MUSIC PLAYER 🎶 🔥 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs 🔰 •

ANKIT KUMAR 1 Dec 27, 2021
Anti Spam/NSFW Telegram Bot Written In Python With Pyrogram.

✨ SpamProtectionRobot ✨ Anti Spam/NSFW Telegram Bot Written In Python With Pyrogram. Requirements Python = 3.7 Install Locally Or On A VPS $ git clon

Akshay Rajput 46 Dec 13, 2022
A multi exploit instagram exploitation framework

Instagram Exploitation Framework About IEF Is an open source Instagram Exploitation Framework with various Exploits that could be used to mod your pro

Instagram Exploitation Framework - BirdSecurity 1 May 23, 2022
Get notifications in your Discord server of any software releases from Apple.

Apple Releases Get notifications in your Discord server of any software releases from Apple. Running To locally host your own instance, create a Disco

adam 17 Oct 22, 2022
Replace sequence_IDs in gff3 based on given genome.fasta

gff-rename Replace the sequence IDs in a gff3 file with a set of provided sequence IDs from a genom.fasta. This is useful when a gff3 file is retrieve

tolkit 1 Nov 12, 2021
Unofficial Python wrapper for official Hacker News API

haxor Unofficial Python wrapper for official Hacker News API. Installation pip install haxor Usage Import and initialization: from hackernews import H

147 Sep 18, 2022
Quickly and efficiently delete your entire tweet history with the help of your Twitter archive without worrying about the pointless 3200 tweet limit imposed by Twitter.

Twitter Nuke Quickly and efficiently delete your entire tweet history with the help of your Twitter archive without worrying about the puny and pointl

Mayur Bhoi 73 Dec 12, 2022
Python library for interacting with the Wunderlist 2 REST API

Overview Wunderpy2 is a thin Python library for accessing the official Wunderlist 2 API. What does a thin library mean here? Only the bare minimum of

mieubrisse 24 Dec 29, 2020
Nonebot2 简易群管

简易群管 ✨ NoneBot2 简易群管 ✨ _ 踢 改 禁 欢迎issue pr 权限说明:permission=SUPERUSER 安装 💿 pip install nonebot-plugin-admin 导入 📲 在bot.py 导入,语句: nonebot.load_plugin("n

幼稚园园长 74 Dec 22, 2022
A basic Ubisoft API wrapper created in python.

UbisoftAPI A basic Ubisoft API wrapper created in python. I will be updating this with more endpoints as time goes on. Please note that this is my fir

Ethan 2 Oct 31, 2021
Crud-python-sqlite: used to manage telephone contacts through python and sqlite

crud-python-sqlite This program is used to manage telephone contacts through python and sqlite. Dependencicas python3 sqlite3 Installation Clone the r

Luis Negrón 0 Jan 24, 2022
A Telegram bot to transcribe audio, video and image into text.

Transcriber Bot A Telegram bot to transcribe audio, video and image into text. Deploy to Heroku Local Deploying Install the FFmpeg. Make sure you have

10 Dec 19, 2022
Portal Backend for Yuta management

Portal Backend for Yuta management Prerequisites Python 3.10 or above. pip, pdm installed. Quickstart Install the required packages: pdm install Runn

Loc Mai 1 Dec 20, 2021