[CVPR'22] COAP: Learning Compositional Occupancy of People

Related tags

Deep LearningCOAP
Overview

COAP: Compositional Articulated Occupancy of People

Paper | Video | Project Page

teaser figure

This is the official implementation of the CVPR 2022 paper COAP: Learning Compositional Occupancy of People.

Description

This repository provides the official implementation of an implicit human body model (COAP) which implements efficient loss terms for resolving self-intersection and collisions with 3D geometries.

Installation

The necessary requirements are specified in the requrements.txt file. To install COAP, execute:

pip install git+https://github.com/markomih/COAP.git

Note that Pytorch3D may require manuall installation (see instructions here). Alternatively, we provide a conda environment file to install the dependences:

conda env create -f environment.yml
conda activate coap
pip install git+https://github.com/markomih/COAP.git

Optional Dependencies

Install the pyrender package to use the visualization/tutorial scripts and follow the additional instructions specified here if you wish to retrain COAP.

Tutorials

COAP extends the interface of the SMPL-X package (follow its instructions for the usage) via two volumetric loss terms: 1) a loss for resolving self-intersections and 2) a loss for resolving collisions with 3D geometries flexibly represented as point clouds. In the following, we provide a minimal interface to access the COAP's functionalities:

import smplx
from coap import attach_coap

# create a SMPL body and extend the SMPL body via COAP (we support: smpl, smplh, and smplx model types)
model = smplx.create(**smpl_parameters)
attach_coap(model)

smpl_output = model(**smpl_data)  # smpl forward pass
# NOTE: make sure that smpl_output contains the valid SMPL variables (pose parameters, joints, and vertices). 
assert model.joint_mapper is None, 'COAP requires valid SMPL joints as input'

# access two loss functions
model.coap.selfpen_loss(smpl_output)  # self-intersections
model.coap.collision_loss(smpl_output, scan_point_cloud)  # collisions with other geometris

Additionally, we provide two tutorials on how to use these terms to resolve self-intersections and collisions with the environment.

Pretrained Models

A respective pretrained model will be automatically fetched and loaded. All the pretrained models are available on the dev branch inside the ./models directory.

Citation

@inproceedings{Mihajlovic:CVPR:2022,
   title = {{COAP}: Compositional Articulated Occupancy of People},
   author = {Mihajlovic, Marko and Saito, Shunsuke and Bansal, Aayush and Zollhoefer, Michael and Tang, Siyu},
   booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
   month = jun,
   year = {2022}
}

Contact

For questions, please contact Marko Mihajlovic ([email protected]) or raise an issue on GitHub.

Owner
Marko Mihajlovic
PhD Student in Computer Vision and Machine Learning at ETH Zurich
Marko Mihajlovic
Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning

Graph-InfoClust-GIC [PAKDD 2021] PAKDD'21 version Graph InfoClust: Maximizing Coarse-Grain Mutual Information in Graphs Preprint version Graph InfoClu

Costas Mavromatis 21 Dec 03, 2022
[3DV 2021] Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation

Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation This is the official implementation for the method described in Ch

Jiaxing Yan 27 Dec 30, 2022
Improving Non-autoregressive Generation with Mixup Training

MIST Training MIST TRAIN_FILE=/your/path/to/train.json VALID_FILE=/your/path/to/valid.json OUTPUT_DIR=/your/path/to/save_checkpoints CACHE_DIR=/your/p

7 Nov 22, 2022
Official code for MPG2: Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN

This is the official code for Multi-attribute Pizza Generator (MPG2): Cross-domain Attribute Control with Conditional StyleGAN. Paper Demo Setup Envir

Fangda Han 5 Sep 01, 2022
Self-Adaptable Point Processes with Nonparametric Time Decays

NPPDecay This is our implementation for the paper Self-Adaptable Point Processes with Nonparametric Time Decays, by Zhimeng Pan, Zheng Wang, Jeff M. P

zpan 2 Sep 24, 2022
Nsdf: A mesh SDF with just some code we can directly paste into our raymarcher

nsdf Representing SDFs of arbitrary meshes has been a bit tricky so far. Express

Jan Ivanecky 5 Feb 18, 2022
UnpNet - Rethinking 3-D LiDAR Point Cloud Segmentation(IEEE TNNLS)

UnpNet Citation Please cite the following paper if you use this repository in your reseach. @article {PMID:34914599, Title = {Rethinking 3-D LiDAR Po

Shijie Li 4 Jul 15, 2022
Revisiting Video Saliency: A Large-scale Benchmark and a New Model (CVPR18, PAMI19)

DHF1K =========================================================================== Wenguan Wang, J. Shen, M.-M Cheng and A. Borji, Revisiting Video Sal

Wenguan Wang 126 Dec 03, 2022
This is an example implementation of the paper "Cross Domain Robot Imitation with Invariant Representation".

IR-GAIL This is an example implementation of the paper "Cross Domain Robot Imitation with Invariant Representation". Dependency The experiments are de

Zhao-Heng Yin 1 Jul 14, 2022
Türkiye Canlı Mobese Görüntülerinde Profesyonel Nesne Takip Sistemi

Türkiye Mobese Görüntü Takip Türkiye Mobese görüntülerinde OPENCV ve Yolo ile takip sistemi Multiple Object Tracking System in Turkish Mobese with OPE

15 Dec 22, 2022
Generalized Random Forests

generalized random forests A pluggable package for forest-based statistical estimation and inference. GRF currently provides non-parametric methods fo

GRF Labs 781 Dec 25, 2022
Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

ACTION-Net Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21). Getting Started EgoGesture data folder struct

V-Sense 171 Dec 26, 2022
Official implementation of NeurIPS'2021 paper TransformerFusion

TransformerFusion: Monocular RGB Scene Reconstruction using Transformers Project Page | Paper | Video TransformerFusion: Monocular RGB Scene Reconstru

Aljaz Bozic 118 Dec 25, 2022
Pure python PEMDAS expression solver without using built-in eval function

pypemdas Pure python PEMDAS expression solver without using built-in eval function. Supports nested parenthesis. Supported operators: + - * / ^ Exampl

1 Dec 22, 2021
Cours d'Algorithmique Appliquée avec Python pour BTS SIO SISR

Course: Introduction to Applied Algorithms with Python (in French) This is the source code of the website for the Applied Algorithms with Python cours

Loic Yvonnet 0 Jan 27, 2022
SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model

SEOVER-Master This code is the implementation of paper: SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model

4 Feb 24, 2022
Neural network-based build time estimation for additive manufacturing

Neural network-based build time estimation for additive manufacturing Oh, Y., Sharp, M., Sprock, T., & Kwon, S. (2021). Neural network-based build tim

Yosep 1 Nov 15, 2021
Two types of Recommender System : Content-based Recommender System and Colaborating filtering based recommender system

Recommender-Systems Two types of Recommender System : Content-based Recommender System and Colaborating filtering based recommender system So the data

Yash Kumar 0 Jan 20, 2022
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Adversarial Robustness Toolbox (ART) is a Python library for Machine Learning Security. ART provides tools that enable developers and researchers to defend and evaluate Machine Learning models and ap

3.4k Jan 04, 2023
BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization

BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization Authors: Wojciech Kryściński, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong,

Salesforce 125 Dec 31, 2022