Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Overview

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

framework

This repository includes the source code for our CVPR 2021 paper on multi-view multi-person 3D pose estimation. Please read our paper for more details at https://arxiv.org/abs/2104.02273. The project webpage is available here.

Bibtex:

@InProceedings{Lin_2021_CVPR,
    author    = {Lin, Jiahao and Lee, Gim Hee},
    title     = {Multi-View Multi-Person 3D Pose Estimation With Plane Sweep Stereo},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {11886-11895}
}

Environment

Our code is tested on

  • Python 3.8.5
  • PyTorch 1.6.0 & torchvision 0.7.0
  • CUDA 11.2

Preparing Data

Download following data before using the code in this repository:

The data should be organized as follows:

    ROOTDIR/
        └── data/
            └── Campus/
                └── actorsGT.mat
                └── calibration_campus.json
                └── pred_campus_maskrcnn_hrnet_coco.pkl
            └── Shelf/
                └── actorsGT.mat
                └── calibration_shelf.json
                └── pred_shelf_maskrcnn_hrnet_coco.pkl
            └── Panoptic/
                └── 160224_haggling1/
                └── 160226_haggling1/
                └── ...
                └── keypoints_train_results.json
                └── keypoints_validation_results.json
            └── panoptic_training_pose.pkl
        └── output/
            └── campus_synthetic/mvmppe/config/model_best_pretrained.pth.tar
            └── shelf_synthetic/mvmppe/config/model_best_pretrained.pth.tar
            └── panoptic/mvmppe/config/model_best_pretrained.pth.tar
        └── ...

Training and Inference

Below are the commands for training our model on different datasets.

The Campus dataset:

    python run/train.py --cfg configs/campus/config.yaml

The Shelf dataset:

    python run/train.py --cfg configs/shelf/config.yaml

The CMU Panoptic dataset:

    python run/train.py --cfg configs/panoptic/config.yaml

Below are the commands for performing inference with our pre-trained models.

The Campus dataset:

    python run/validate.py --cfg configs/campus/config.yaml -t pretrained

The Shelf dataset:

    python run/validate.py --cfg configs/shelf/config.yaml -t pretrained

The CMU Panoptic dataset:

    python run/validate.py --cfg configs/panoptic/config.yaml -t pretrained
Owner
Jiahao Lin
Jiahao Lin
A simple baseline for 3d human pose estimation in PyTorch.

3d_pose_baseline_pytorch A PyTorch implementation of a simple baseline for 3d human pose estimation. You can check the original Tensorflow implementat

weigq 312 Jan 06, 2023
an implementation of 3D Ken Burns Effect from a Single Image using PyTorch

3d-ken-burns This is a reference implementation of 3D Ken Burns Effect from a Single Image [1] using PyTorch. Given a single input image, it animates

Simon Niklaus 1.4k Dec 28, 2022
PyElecCL - Electron Monte Carlo Second Checks

PyElecCL Python program to perform second checks for electron Monte Carlo radiat

Reese Haywood 3 Feb 22, 2022
1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

Lihe Yang 209 Jan 01, 2023
Convolutional Neural Network for Text Classification in Tensorflow

This code belongs to the "Implementing a CNN for Text Classification in Tensorflow" blog post. It is slightly simplified implementation of Kim's Convo

Denny Britz 5.5k Jan 02, 2023
Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN

Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN If you use this code for your research, please cite ou

41 Dec 08, 2022
Implementation of SETR model, Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.

SETR - Pytorch Since the original paper (Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.) has no official

zhaohu xing 112 Dec 16, 2022
Official implementation of the paper "AAVAE: Augmentation-AugmentedVariational Autoencoders"

AAVAE Official implementation of the paper "AAVAE: Augmentation-AugmentedVariational Autoencoders" Abstract Recent methods for self-supervised learnin

Grid AI Labs 48 Dec 12, 2022
Use VITS and Opencpop to develop singing voice synthesis; Maybe it will VISinger.

Init Use VITS and Opencpop to develop singing voice synthesis; Maybe it will VISinger. 本项目基于 https://github.com/jaywalnut310/vits https://github.com/S

AmorTX 107 Dec 23, 2022
Pytorch implementation of our method for regularizing nerual radiance fields for few-shot neural volume rendering.

InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering Pytorch implementation of our method for regularizing nerual radiance fields f

106 Jan 06, 2023
UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model

UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model Official repository for the ICCV 2021 paper: UltraPose: Syn

MomoAILab 92 Dec 21, 2022
Build Graph Nets in Tensorflow

Graph Nets library Graph Nets is DeepMind's library for building graph networks in Tensorflow and Sonnet. Contact DeepMind 5.2k Jan 05, 2023

Official PyTorch implementation of Data-free Knowledge Distillation for Object Detection, WACV 2021.

Introduction This repository is the official PyTorch implementation of Data-free Knowledge Distillation for Object Detection, WACV 2021. Data-free Kno

NVIDIA Research Projects 50 Jan 05, 2023
Lex Rosetta: Transfer of Predictive Models Across Languages, Jurisdictions, and Legal Domains

Lex Rosetta: Transfer of Predictive Models Across Languages, Jurisdictions, and Legal Domains This is an accompanying repository to the ICAIL 2021 pap

4 Dec 16, 2021
MvtecAD unsupervised Anomaly Detection

MvtecAD unsupervised Anomaly Detection This respository is the unofficial implementations of DFR: Deep Feature Reconstruction for Unsupervised Anomaly

0 Feb 25, 2022
Alex Pashevich 62 Dec 24, 2022
Keras Image Embeddings using Contrastive Loss

Image to Embedding projection in vector space. Implementation in keras and tensorflow of batch all triplet loss for one-shot/few-shot learning.

Shravan Anand K 5 Mar 21, 2022
A list of multi-task learning papers and projects.

This page contains a list of papers on multi-task learning for computer vision. Please create a pull request if you wish to add anything. If you are interested, consider reading our recent survey pap

svandenh 297 Dec 17, 2022
Simple Python application to transform Serial data into OSC messages

SerialToOSC-Bridge Simple Python application to transform Serial data into OSC messages. The current purpose is to be a compatibility layer between ha

Division of Applied Acoustics at Chalmers University of Technology 3 Jun 03, 2021
Embracing Single Stride 3D Object Detector with Sparse Transformer

SST: Single-stride Sparse Transformer This is the official implementation of paper: Embracing Single Stride 3D Object Detector with Sparse Transformer

TuSimple 385 Dec 28, 2022