Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Last update: Oct 27, 2022

Overview

Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking
Hau Chu, Jia-Hong Lee, Yao-Chih Lee, Ching-Hsien Hsu, Jia-Da Li, Chu-Song Chen
2021 CVPR B-AMFG Workshop

Note: It's a project of AI^2 Lab. The code will be update in here while there is a new version.

Installation

Python 3.6+
Cuda 9.0
Cudnn 7
gcc 5 & g++ 5 (for Ubuntu 18.04)

$ sudo apt install gcc-5 g++-5
$ sudo ln -s /usr/bin/gcc-6 /usr/local/bin/gcc
$ sudo ln -s /usr/bin/g++-6 /usr/local/bin/g++

Conda Env

$ conda create -n venv python=3.6
$ conda activate venv
$ conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=9.0 -c pytorch
$ pip install tensorflow_gpu==1.9.0
$ pip install -r requirements.txt

$ sudo apt install git

Data preparation

Download datasets:

Campus (http://campar.in.tum.de/Chair/MultiHumanPose)
Shelf (http://campar.in.tum.de/Chair/MultiHumanPose)
CMU Panoptic (https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox)

Dataset's camera_parameter.pickle download

The directory tree should look like below:

${ROOT}
    |-- CatchImage
        |-- CampusSeq1
        |   |-- Camera0
        |   |-- Camera1
        |   |-- Camera2
        |   |-- camera_parameter.pickle
        |   |-- actorsGT.mat
        |-- Shelf
        |   |-- Camera0
        |   |-- ...
        |   |-- Camera4
        |   |-- camera_parameter.pickle
        |   |-- actorsGT.mat
        |-- Panoptic
        |   |-- 160906_pizza1
            |   |-- 00_03 # hdImgs folder of 03 camera
            |   |-- 00_06 # hdImgs folder of 06 camera
            |   |-- ...
            |   |-- camera_parameter.pickle
            |   |-- hdPose_stage1_coco19
            |-- ...
    |-- src

Backend Models

Backend models, which is not our works, are released codes from others. We only did some small modifications to fit the format of our input/output. Put models in {ROOT}/src/backend

YOLOv3
- origin: https://github.com/ZQPei/deep_sort_pytorch
- modification
HRNet
- origin: https://github.com/stefanopini/simple-HRNet
- modification

Run Codes

Demo

$cd src
python -W ignore testmodel.py --dataset CampusSeq1 # For Campus
python -W ignore testmodel.py --dataset Shelf # For Shelf
python -W ignore testmodel.py --dataset Panoptic # For Panoptic (sub-dataset can be modified in config)

Evaluation

$cd src
python -W ignore evalmodel.py --dataset CampusSeq1 
python -W ignore evalmodel.py --dataset Shelf

Campus PCP Score

Bone Group	Actor 0	Actor 1	Actor 2	Average
Head	100.00	100.00	100.00	100.00
Torso	100.00	100.00	100.00	100.00
Upper arms	98.98	100.00	100.00	99.66
Lower arms	92.86	68.78	91.30	84.31
Upper legs	100.00	100.00	100.00	100.00
Lower legs	100.00	100.00	100.00	100.00
Total	98.37	93.76	98.26	96.79

Shelf PCP Score

Bone Group	Actor 0	Actor 1	Actor 2	Average
Head	94.98	100.00	91.30	95.43
Torso	100.00	100.00	100.00	100.00
Upper arms	100.00	100.00	96.27	98.76
Lower arms	98.21	77.03	96.27	90.50
Upper legs	100.00	100.00	100.00	100.00
Lower legs	100.00	100.00	100.00	100.00
Total	99.14	95.41	97.64	97.39

Citation

@InProceedings{Chu_2021_CVPR,
    author    = {Chu, Hau and Lee, Jia-Hong and Lee, Yao-Chih and Hsu, Ching-Hsien and Li, Jia-Da and Chen, Chu-Song},
    title     = {Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2021},
    pages     = {1472-1481}
}

Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Related tags

Overview

Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Installation

Data preparation

Backend Models

Run Codes

Demo

Evaluation

Campus PCP Score

Shelf PCP Score

Citation

Owner

Retrieve and analysis data from SDSS (Sloan Digital Sky Survey)

Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

Filtering variational quantum algorithms for combinatorial optimization

Face recognition system using MTCNN, FACENET, SVM and FAST API to track participants of Big Brother Brasil in real time.

Multimodal Co-Attention Transformer (MCAT) for Survival Prediction in Gigapixel Whole Slide Images

Code for the paper "TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks"

BRNet - code for Automated assessment of BI-RADS categories for ultrasound images using multi-scale neural networks with an order-constrained loss function

An implementation of an abstract algebra for music tones (pitches).

Welcome to The Eigensolver Quantum School, a quantum computing crash course designed by students for students.

🧠 A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation.', ECCV 2016

Open-source implementation of Google Vizier for hyper parameters tuning

AdaFocus (ICCV 2021) Adaptive Focus for Efficient Video Recognition

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization

Robotics with GPU computing

Convert Table data to approximate values with GUI

Learning Visual Words for Weakly-Supervised Semantic Segmentation

Code for "Retrieving Black-box Optimal Images from External Databases" (WSDM 2022)

MDMM - Learning multi-domain multi-modality I2I translation

OpenMMLab Semantic Segmentation Toolbox and Benchmark.