[CVPR'22] COAP: Learning Compositional Occupancy of People

Related tags

Deep LearningCOAP
Overview

COAP: Compositional Articulated Occupancy of People

Paper | Video | Project Page

teaser figure

This is the official implementation of the CVPR 2022 paper COAP: Learning Compositional Occupancy of People.

Description

This repository provides the official implementation of an implicit human body model (COAP) which implements efficient loss terms for resolving self-intersection and collisions with 3D geometries.

Installation

The necessary requirements are specified in the requrements.txt file. To install COAP, execute:

pip install git+https://github.com/markomih/COAP.git

Note that Pytorch3D may require manuall installation (see instructions here). Alternatively, we provide a conda environment file to install the dependences:

conda env create -f environment.yml
conda activate coap
pip install git+https://github.com/markomih/COAP.git

Optional Dependencies

Install the pyrender package to use the visualization/tutorial scripts and follow the additional instructions specified here if you wish to retrain COAP.

Tutorials

COAP extends the interface of the SMPL-X package (follow its instructions for the usage) via two volumetric loss terms: 1) a loss for resolving self-intersections and 2) a loss for resolving collisions with 3D geometries flexibly represented as point clouds. In the following, we provide a minimal interface to access the COAP's functionalities:

import smplx
from coap import attach_coap

# create a SMPL body and extend the SMPL body via COAP (we support: smpl, smplh, and smplx model types)
model = smplx.create(**smpl_parameters)
attach_coap(model)

smpl_output = model(**smpl_data)  # smpl forward pass
# NOTE: make sure that smpl_output contains the valid SMPL variables (pose parameters, joints, and vertices). 
assert model.joint_mapper is None, 'COAP requires valid SMPL joints as input'

# access two loss functions
model.coap.selfpen_loss(smpl_output)  # self-intersections
model.coap.collision_loss(smpl_output, scan_point_cloud)  # collisions with other geometris

Additionally, we provide two tutorials on how to use these terms to resolve self-intersections and collisions with the environment.

Pretrained Models

A respective pretrained model will be automatically fetched and loaded. All the pretrained models are available on the dev branch inside the ./models directory.

Citation

@inproceedings{Mihajlovic:CVPR:2022,
   title = {{COAP}: Compositional Articulated Occupancy of People},
   author = {Mihajlovic, Marko and Saito, Shunsuke and Bansal, Aayush and Zollhoefer, Michael and Tang, Siyu},
   booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
   month = jun,
   year = {2022}
}

Contact

For questions, please contact Marko Mihajlovic ([email protected]) or raise an issue on GitHub.

Owner
Marko Mihajlovic
PhD Student in Computer Vision and Machine Learning at ETH Zurich
Marko Mihajlovic
Code for the paper "M2m: Imbalanced Classification via Major-to-minor Translation" (CVPR 2020)

M2m: Imbalanced Classification via Major-to-minor Translation This repository contains code for the paper "M2m: Imbalanced Classification via Major-to

79 Oct 13, 2022
PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

Authors official PyTorch implementation of the "WarpedGANSpace: Finding non-linear RBF paths in GAN latent space" [ICCV 2021].

Christos Tzelepis 100 Dec 06, 2022
Simple and Distributed Machine Learning

Synapse Machine Learning SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. Sy

Microsoft 3.9k Dec 30, 2022
Fast and scalable uncertainty quantification for neural molecular property prediction, accelerated optimization, and guided virtual screening.

Evidential Deep Learning for Guided Molecular Property Prediction and Discovery Ava Soleimany*, Alexander Amini*, Samuel Goldman*, Daniela Rus, Sangee

Alexander Amini 75 Dec 15, 2022
Depression Asisstant GDSC Challenge Solution

Depression Asisstant can help you give solution. Please using Python version 3.9.5 for contribute.

Ananda Rauf 1 Jan 30, 2022
DeOldify - A Deep Learning based project for colorizing and restoring old images (and video!)

DeOldify - A Deep Learning based project for colorizing and restoring old images (and video!)

Jason Antic 15.8k Jan 04, 2023
This code is a toolbox that uses Torch library for training and evaluating the ERFNet architecture for semantic segmentation.

ERFNet This code is a toolbox that uses Torch library for training and evaluating the ERFNet architecture for semantic segmentation. NEW!! New PyTorch

Edu 104 Jan 05, 2023
PyTorch implementation for NED. It can be used to manipulate the facial emotions of actors in videos based on emotion labels or reference styles.

Neural Emotion Director (NED) - Official Pytorch Implementation Example video of facial emotion manipulation while retaining the original mouth motion

Foivos Paraperas 89 Dec 23, 2022
Using Self-Supervised Pretext Tasks for Active Learning - Official Pytorch Implementation

Using Self-Supervised Pretext Tasks for Active Learning - Official Pytorch Implementation Experiment Setting: CIFAR10 (downloaded and saved in ./DATA

John Seon Keun Yi 38 Dec 27, 2022
The code for the NeurIPS 2021 paper "A Unified View of cGANs with and without Classifiers".

Energy-based Conditional Generative Adversarial Network (ECGAN) This is the code for the NeurIPS 2021 paper "A Unified View of cGANs with and without

sianchen 22 May 28, 2022
Demo for Real-time RGBD-based Extended Body Pose Estimation paper

Real-time RGBD-based Extended Body Pose Estimation This repository is a real-time demo for our paper that was published at WACV 2021 conference The ou

Renat Bashirov 118 Dec 26, 2022
BiSeNet based on pytorch

BiSeNet BiSeNet based on pytorch 0.4.1 and python 3.6 Dataset Download CamVid dataset from Google Drive or Baidu Yun(6xw4). Pretrained model Download

367 Dec 26, 2022
Scalable training for dense retrieval models.

Scalable implementation of dense retrieval. Training on cluster By default it trains locally: PYTHONPATH=.:$PYTHONPATH python dpr_scale/main.py traine

Facebook Research 90 Dec 28, 2022
The repository contain code for building compiler using puthon.

Building Compiler This is a python implementation of JamieBuild's "Super Tiny Compiler" Overview JamieBuilds developed a wonderfully educative compile

Shyam Das Shrestha 1 Nov 21, 2021
Detecting drunk people through thermal images using Deep Learning (CNN)

Drunk Detection CNN Detecting drunk people through thermal images using Deep Learning (CNN) Dataset We used thermal images provided by Electronics Lab

Giacomo Ferretti 3 Oct 27, 2022
Residual Dense Net De-Interlace Filter (RDNDIF)

Residual Dense Net De-Interlace Filter (RDNDIF) Work in progress deep de-interlacer filter. It is based on the architecture proposed by Bernasconi et

Louis 7 Feb 15, 2022
MG-GCN: Scalable Multi-GPU GCN Training Framework

MG-GCN MG-GCN: multi-GPU GCN training framework. For more information, please read our paper. After cloning our repository, run git submodule update -

Translational Data Analytics (TDA) Lab @GaTech 6 Oct 24, 2022
A python program to hack instagram

hackinsta a program to hack instagram Yokoback_(instahack) is the file to open, you need libraries write on import. You run that file in the same fold

2 Jan 22, 2022
Scientific Computation Methods in C and Python (Open for Hacktoberfest 2021)

Sci - cpy README is a stub. Do expand it. Objective This repository is meant to be a ready reference for scientific computation methods. Do ⭐ it if yo

Sandip Dutta 7 Oct 12, 2022
(Py)TOD: Tensor-based Outlier Detection, A General GPU-Accelerated Framework

(Py)TOD: Tensor-based Outlier Detection, A General GPU-Accelerated Framework Background: Outlier detection (OD) is a key data mining task for identify

Yue Zhao 127 Jan 05, 2023