Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
AWS Glue PySpark - Apache Hudi Quick Start Guide

AWS Glue PySpark - Apache Hudi Quick Start Guide Disclaimer: This is a quick start guide for the Apache Hudi Python Spark connector, running on AWS Gl

Gabriel Amazonas Mesquita 8 Nov 14, 2022
Instant messaging client in tkinter

Concord_client_tk Instant messaging client in tkinter Contributors : Ilade-s [https://github.com/Ilade-s] Doku [https://github.com/D0kuhebi] Descripti

Raphaël Merlet 2 Jun 15, 2022
A python library for creating Slack slash commands using AWS Lambda Functions

slashbot Slashbot makes it easy to create slash commands using AWS Lambda functions. These can be handy for creating a secure way to execute automated

Eric Brassell 17 Oct 21, 2022
A simple python discord bot with commands for moderation and utility.

Discord Bot A simple python discord bot with commands for moderation, utility and fun. Moderation $kick user reason - Kick a user from the server

3 Jan 07, 2022
And now, for the first time, you can send alerts via action from ArcSight ESM Console to the TheHive when Correlation Rules are triggered.

ArcSight Integration with TheHive And now, for the first time, you can send alerts via action from ArcSight ESM Console to the TheHive when Correlatio

Amir Hossein Zargaran 3 Jan 19, 2022
A simple anti-ghostping python bot made using diskord.

Anti Ghostping A simple Anti-Ghostping python bot made with ❤ using Diskord Requirements No one will use this but, all you need for this bot is: Pytho

RyZe 2 Sep 12, 2022
1 Feb 18, 2022
Customizable and open-sourced bot for a few private servers

MarlBot A private bot for controlling monkeys and turtles. Why does this bot exist? The bot exists as a general-purpose community bot for a select few

KR 1 Jan 18, 2022
Python Library for Secp256k1 Bitcoin curve to do fast ECC calculation

secp256k1 Python Library for Secp256k1 Bitcoin curve to do fast ECC calculation Example Usage import secp256k1 as ice print('[C]',privatekey_to_addres

iceland 49 Jan 01, 2023
Clash of Clans developer unofficial api Wrapper to generate ip based token

Clash of Clans developer unofficial api Wrapper to generate ip based token

Aryan Vikash 6 Apr 01, 2022
This is a straightforward python implementation to specifically grab basic infos about IPO companies in China from Sina Stock website.

SinaStockBasicInfoCollect This is a straightforward python implementation to specifically grab basic infos about IPO companies in China from Sina Stoc

CrosSea 1 Dec 09, 2021
Dashbot is an application for showing the trade in U.S. Trade Market

Dashbot is an application for showing the trade in U.S. Trade Market (e.g., 4:00am to 8:00pm Eastern Time for the US market using Intraday data using Mercury..

Ahmed Nabil 2 Jan 27, 2022
Async ShareX uploader written in python

Async ShareX uploader written in python

Jacob 2 Jan 07, 2022
Injector/automatic translator (using deepL API) for Tsukihime Remake

deepLuna Extractor/Editor/Translator/Injector for Tsukihime Remake About deepLuna, from "deepL", the machine translation service, and "Luna", the name

30 Dec 15, 2022
A python script for hitting the kik API to enumerate people based on a username/userlist

kick3d Recon script for enumerating users off of the Kik API. This script has the ability to check single usernames or run through a userlist of usern

Sakura Samurai 19 Oct 04, 2021
A bot that downloads all the necessary files from WeLearn and lists your assignments, filter due assignments, etc.

Welearn-bot This is a bot which lets you interact with WeLearn from the command line. It can Download all files/resources from your courses and organi

Parth Bibekar 17 Oct 19, 2022
An example Music Bot written in Disnake and uses slash commands to operate.

Music Bot An example music bot that is written in Disnake [Maintained discord.py Fork] Disnake Disnake is a maintained and updated fork of discord.py.

6 Jan 08, 2022
A supabase client for python

supabase-client A Supabase client for Python. This mirrors the design of supabase-js Full documentation: https://keosariel.github.io/2021/08/08/supaba

kenneth gabriel 11 Dec 19, 2022
Binance leverage futures Hook

Simple binance futures Attention Just use leverage. The fee difference between futures and spot is not considered. For example, funding rate, etc. Onl

Adriance 26 Aug 27, 2022
streamlit translator is used to detect and translate between languages created using gTTS, googletrans, pillow and streamlit python packages

Streamlit Translator Streamlit Translator is a simple translator app to detect and translate between languages. Streamlit Translator gets text and lan

Siva Prakash 5 Apr 05, 2022