Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
python library to the bitly api

bitly API python library Installation pip install bitly_api Run tests Your username is the lowercase name shown when you login to bitly, your access

Bitly 245 Aug 14, 2022
Twitter-Scrapping - Tweeter tweets extracting using python

Twitter-Scrapping Twitter tweets extracting using python This project is to extr

Suryadeepsinh Gohil 2 Feb 04, 2022
Create custom Vanity URLs for Discord without 30 boosts

CustomVanity - Made by udp#6666 aka Apolo - OpenSource Custom Discord Vanity Creator How To Use Open CustomVanity.py Write your server invite code Wri

apolo 17 Aug 23, 2022
A Python wrapper around the Twitter API.

Python Twitter A Python wrapper around the Twitter API. By the Python-Twitter Developers Introduction This library provides a pure Python interface fo

Mike Taylor 3.4k Jan 01, 2023
It is a temporary project to study discord interactions. You can set permissions conveniently when you invite a particular disk code bot.

Permission Bot 디스코드 내에 있는 message-components 를 연구하기 위하여 제작된 봇입니다. Setup /config/config_example.ini 파일을 /config/config.ini으로 변환합니다. config 파일의 기본 양식은 아

gunyu1019 4 Mar 07, 2022
Spotify Web API client for Python 3

Welcome to the GitHub repository of Tekore! We provide a client for the Spotify Web API for Python, complete with all available endpoints and authenti

Felix Hildén 186 Dec 22, 2022
A way to export your saved reddit posts to a Notion table.

reddit-saved-to-notion A way to export your saved reddit posts and comments to a Notion table.Uses notion-sdk-py and praw for interacting with Notion

19 Sep 12, 2022
PESU Academy Discord Bot built for PESsants and PESts of PES University

PESU Academy Bot PESU Academy Discord Bot built for PESsants and PESts of PES University You can add the bot to your Discord Server using this link. O

Aditeya Baral 0 Nov 16, 2021
Easily report Instagram pages and close the page

Program Features - 📌 Delete target post on Instagram. - 📌 Delete Media Target post on Instagram - 📌 Complete deletion of the target account on Inst

hack4lx 11 Nov 25, 2022
To send an Instagram message using Python

To send an Instagram message using Python, you must have an Instagram account and install the Instabot library in your Python virtual environment.

Coding Taggers 1 Dec 18, 2021
Bootstrapping your personal Web3 info hub from more than 500 RSS Feeds.

RSS Aggregator for Web3 (or 🥩 RAW for short) Bootstrapping your personal Web3 info hub from more than 500 RSS Feeds. What is RSS or Reader Services?

ChainFeeds 1.8k Dec 29, 2022
OAN Music - Highly advanced User Music Bot

ཧᜰ꙰ꦿ➢𝐎𝐀𝐍༒☛ 🎧 Advanced 𝐎𝐀𝐍 Music bot. 🔗 𝐏𝐨𝐰𝐞𝐫𝐞𝐝 𝐛𝐲 : ➢𝐀ttitude

Attitude king 5 Feb 25, 2022
A simple but useful Discord Selfbot with essential features, made with discord.py-self.

Discord Selfbot Xyno Discord Selfbot Xyno is a simple but useful selfbot for Discord. It has currently limited useful features but it will be updated

Amit Pathak 7 Apr 24, 2022
Boilerplate template for the discord-py-interactions library

discord-py-interactions_boilerplate Boilerplate template for the discord-py-interactions library Currently, this boilerplate supports discord-py-inter

Ventus 7 Dec 03, 2022
QR-Code-Grabber - A python script that allows a person to create a qr code token grabber

Qr Code Grabber Description Un script python qui permet a une personne de creer

5 Jun 28, 2022
Repository for the IPvSeeYou talk at Black Hat 2021

IPvSeeYou Geolocation Lookup Tool Overview IPvSeeYou.py is a tool to assist with geolocating EUI-64 IPv6 hosts. It takes as input an EUI-64-derived MA

57 Nov 08, 2022
This bot will send you an email or notify you via telegram & discord if dolar/lira parity breaks a record.

Dolar Rekor Kırdı Mı? This bot will send you an email or notify you via Telegram & Discord if Dolar/Lira parity breaks a record. Mailgun can be used a

Yiğit Göktuğ Budanur 2 Oct 14, 2021
Mega.nz to GDrive uploader

Mega.nz to GDrive uploader With this telegram bot you can download files from mega.nz and upload those files or telegram uploaded files to GDrive. You

30 Nov 13, 2022
Policy and data administration, distribution, and real-time updates on top of Open Policy Agent

⚡ OPAL ⚡ Open Policy Administration Layer OPAL is an administration layer for Open Policy Agent (OPA), detecting changes to both policy and policy dat

8 Dec 07, 2022
🤖 The bot that runs the official Fairfield Programming Association Discord server.

🤖 The bot that runs the official Fairfield Programming Association Discord server.

Fairfield Programming Association 1 Jan 07, 2022