Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A discord self-bot to automate shitposting for your everyday needs.

Shitpost Selfbot A discord self-bot to automate shitposting for your everyday needs. Caution: May be a little racist. I have no clue where we are taki

stormy 1 Mar 31, 2022
Change your discord avatar every x h/d based on a list of images

Discord-Avatar-Autochange Introduction A simple script that automatically keeps changing your discord avatar after a given amount of time based on the

Armin Amiri 5 Apr 30, 2022
A Terminal User Interface (TUI) for automated trading with Komodo Platform's AtomicDEX-API

PytomicDEX Makerbot A Terminal User Interface (TUI) for automated trading with Komodo Platform's AtomicDEX-API Install sudo apt install wget curl jq g

6 Aug 25, 2022
A Bot To Get Info Of Telegram messages , Media , Channel id Group ID etc.

Info-Bot A Bot To Get Info Of Telegram messages , Media , Channel id Group ID etc. Get Info Of Your And Messages , Channels , Groups ETC... How to mak

Vɪᴠᴇᴋ 23 Nov 12, 2022
Esse script procura qualquer, dados que você queira na wikipedia! Em breve traremos um com dados em toda a internet.

Buscador de dados simples Dependências necessárias Para você poder começar a utilizar esta ferramenta, você vai precisar da dependência "wikipedia", p

Erick Campoy 4 Feb 24, 2022
The official Python library for Shodan

shodan: The official Python library and CLI for Shodan Shodan is a search engine for Internet-connected devices. Google lets you search for websites,

John Matherly 2.1k Dec 31, 2022
The official Python library for the Plutto API

Plutto Ruby SDK This library will help you easily integrate Plutto API to your software, making your developer life a little bit more enjoyable. Insta

Plutto 3 Nov 23, 2021
A discord token grabber made in Python 3

Discord Token Grabber A Discord token grabber written in Python 3. This version of the grabber only supports Windows. Features Transfers via Discord w

Mega145 4 Aug 04, 2022
A simple, fast, and awesome discord nuke bot! The only thing you need to add is your bot token.

SimpleNukeBot A simple, fast, and awesome discord nuke bot! The only thing you need to add is your bot token. Instructions: All you need to do is crea

Bisc 1 Apr 18, 2022
A solution designed to extract, transform and load Chicago crime data from an RDS instance to other services in AWS.

This project is intended to implement a solution designed to extract, transform and load Chicago crime data from an RDS instance to other services in AWS.

Yesaswi Avula 1 Feb 04, 2022
1 Feb 18, 2022
Twitter FakeNFT With Python

This project is a server that fetches your Twitter profile picture and applies the hexagonal transparency mask displayed on the profiles of users who have an NFT profile picture.

Mathis HAMMEL 29 Apr 23, 2022
Watches your earnings on EarnApp and notifies you when you earned balance or received an payout.

EarnApp-Earning-Monitor Watches your earnings on EarnApp and notifies you when you earned balance or received an payout. Installation Install Python3

Yariya 21 Oct 17, 2022
Python based Spotify account generator.

Spotify Account Generator Python based Spotify account generator. Installation Download the latest release, open command prompt in the folder, run pip

polo 5 Dec 27, 2022
Wetterdienst - Open weather data for humans

We are a group of like-minded people trying to make access to weather data in Python feel like a warm summer breeze, similar to other projects like rdwd for the R language, which originally drew our

226 Jan 04, 2023
OpenSource bot for control groups ...

⭕️ کمک به افراد برای اداره هرچه فان تره گروه 📟 همه گروه های بزرگ نیاز به یه بات خفن دارن تا از گروه مراقبت کنه این بات کارش همینه سعی کرده فیچر خیلی

Mehran Alam Beigi 2 Nov 26, 2021
Open API to list Viet Nam administrative divisions

Viet Nam province API Homepage: https://provinces.open-api.vn This is online tool to let my VietnamProvinces library reach more users. VietnamProvince

Nguyễn Hồng Quân 52 Dec 05, 2022
thumbor is an open-source photo thumbnail service by globo.com

Survey If you use thumbor, please take 1 minute and answer this survey? It's only 2 questions and one is multiple choice!!! thumbor is a smart imaging

Thumbor (by @globocom) 9.3k Dec 31, 2022
A Characther powerful in saints saiya anime and modular telegram group management bot built using python3

Kaneki Ken A Powerful and Modular Saint Aries is a Characther powerful in saints saiya anime and modular telegram group management bot built using pyt

1 Dec 21, 2021
A complete Python application to automatize the process of uploading files to Amazon S3

Upload files or folders (even with subfolders) to Amazon S3 in a totally automatized way taking advantage of: Amazon S3 Multipart Upload: The uploaded

Pol Alzina 1 Nov 20, 2021