[CVPR'22] COAP: Learning Compositional Occupancy of People

Related tags

Deep LearningCOAP
Overview

COAP: Compositional Articulated Occupancy of People

Paper | Video | Project Page

teaser figure

This is the official implementation of the CVPR 2022 paper COAP: Learning Compositional Occupancy of People.

Description

This repository provides the official implementation of an implicit human body model (COAP) which implements efficient loss terms for resolving self-intersection and collisions with 3D geometries.

Installation

The necessary requirements are specified in the requrements.txt file. To install COAP, execute:

pip install git+https://github.com/markomih/COAP.git

Note that Pytorch3D may require manuall installation (see instructions here). Alternatively, we provide a conda environment file to install the dependences:

conda env create -f environment.yml
conda activate coap
pip install git+https://github.com/markomih/COAP.git

Optional Dependencies

Install the pyrender package to use the visualization/tutorial scripts and follow the additional instructions specified here if you wish to retrain COAP.

Tutorials

COAP extends the interface of the SMPL-X package (follow its instructions for the usage) via two volumetric loss terms: 1) a loss for resolving self-intersections and 2) a loss for resolving collisions with 3D geometries flexibly represented as point clouds. In the following, we provide a minimal interface to access the COAP's functionalities:

import smplx
from coap import attach_coap

# create a SMPL body and extend the SMPL body via COAP (we support: smpl, smplh, and smplx model types)
model = smplx.create(**smpl_parameters)
attach_coap(model)

smpl_output = model(**smpl_data)  # smpl forward pass
# NOTE: make sure that smpl_output contains the valid SMPL variables (pose parameters, joints, and vertices). 
assert model.joint_mapper is None, 'COAP requires valid SMPL joints as input'

# access two loss functions
model.coap.selfpen_loss(smpl_output)  # self-intersections
model.coap.collision_loss(smpl_output, scan_point_cloud)  # collisions with other geometris

Additionally, we provide two tutorials on how to use these terms to resolve self-intersections and collisions with the environment.

Pretrained Models

A respective pretrained model will be automatically fetched and loaded. All the pretrained models are available on the dev branch inside the ./models directory.

Citation

@inproceedings{Mihajlovic:CVPR:2022,
   title = {{COAP}: Compositional Articulated Occupancy of People},
   author = {Mihajlovic, Marko and Saito, Shunsuke and Bansal, Aayush and Zollhoefer, Michael and Tang, Siyu},
   booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
   month = jun,
   year = {2022}
}

Contact

For questions, please contact Marko Mihajlovic ([email protected]) or raise an issue on GitHub.

Owner
Marko Mihajlovic
PhD Student in Computer Vision and Machine Learning at ETH Zurich
Marko Mihajlovic
Activity tragle - Google is tracking everything, we just look at it

activity_tragle Google is tracking everything, we just look at it here. You need

BERNARD Guillaume 1 Feb 15, 2022
Malmo Collaborative AI Challenge - Team Pig Catcher

The Malmo Collaborative AI Challenge - Team Pig Catcher Approach The challenge involves 2 agents who can either cooperate or defect. The optimal polic

Kai Arulkumaran 66 Jun 29, 2022
Multi-Anchor Active Domain Adaptation for Semantic Segmentation (ICCV 2021 Oral)

Multi-Anchor Active Domain Adaptation for Semantic Segmentation Munan Ning*, Donghuan Lu*, Dong Wei†, Cheng Bian, Chenglang Yuan, Shuang Yu, Kai Ma, Y

Munan Ning 36 Dec 07, 2022
EfficientNetV2-with-TPU - Cifar-10 case study

EfficientNetV2-with-TPU EfficientNet EfficientNetV2 adalah jenis jaringan saraf convolutional yang memiliki kecepatan pelatihan lebih cepat dan efisie

Sultan syach 1 Dec 28, 2021
Leaderboard, taxonomy, and curated list of few-shot object detection papers.

Leaderboard, taxonomy, and curated list of few-shot object detection papers.

Gabriel Huang 70 Jan 07, 2023
SelfRemaster: SSL Speech Restoration

SelfRemaster: Self-Supervised Speech Restoration Official implementation of SelfRemaster: Self-Supervised Speech Restoration with Analysis-by-Synthesi

Takaaki Saeki 46 Jan 07, 2023
Keras udrl - Keras implementation of Upside Down Reinforcement Learning

keras_udrl Keras implementation of Upside Down Reinforcement Learning This is me

Eder Santana 7 Jan 24, 2022
Data, model training, and evaluation code for "PubTables-1M: Towards a universal dataset and metrics for training and evaluating table extraction models".

PubTables-1M This repository contains training and evaluation code for the paper "PubTables-1M: Towards a universal dataset and metrics for training a

Microsoft 365 Jan 04, 2023
A simple implementation of Kalman filter in Multi Object Tracking

kalman Filter in Multi-object Tracking A simple implementation of Kalman filter in Multi Object Tracking 本实现是在https://github.com/liuchangji/kalman-fil

124 Dec 29, 2022
Learning embeddings for classification, retrieval and ranking.

StarSpace StarSpace is a general-purpose neural model for efficient learning of entity embeddings for solving a wide variety of problems: Learning wor

Facebook Research 3.8k Dec 22, 2022
Running AlphaFold2 (from ColabFold) in Azure Machine Learning

Running AlphaFold2 (from ColabFold) in Azure Machine Learning Colby T. Ford, Ph.D. Companion repository for Medium Post: How to predict many protein s

Colby T. Ford 3 Feb 18, 2022
Material for my PyConDE & PyData Berlin 2022 Talk "5 Steps to Speed Up Your Data-Analysis on a Single Core"

5 Steps to Speed Up Your Data-Analysis on a Single Core Material for my talk at the PyConDE & PyData Berlin 2022 Description Your data analysis pipeli

Jonathan Striebel 9 Dec 12, 2022
A curated list of Machine Learning and Deep Learning tutorials in Jupyter Notebook format ready to run in Google Colaboratory

Awesome Machine Learning Jupyter Notebooks for Google Colaboratory A curated list of Machine Learning and Deep Learning tutorials in Jupyter Notebook

Carlos Toxtli 245 Jan 01, 2023
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

Ran Cheng 4 Dec 15, 2022
Baseline of DCASE 2020 task 4

Couple Learning for SED This repository provides the data and source code for sound event detection (SED) task. The improvement of the Couple Learning

21 Oct 18, 2022
A GPU-optional modular synthesizer in pytorch, 16200x faster than realtime, for audio ML researchers.

torchsynth The fastest synth in the universe. Introduction torchsynth is based upon traditional modular synthesis written in pytorch. It is GPU-option

torchsynth 229 Jan 02, 2023
Pansharpening by convolutional neural networks in the full resolution framework

Z-PNN: Zoom Pansharpening Neural Network Pansharpening by convolutional neural networks in the full resolution framework is a deep learning method for

20 Nov 24, 2022
Real-time multi-object tracker using YOLO v5 and deep sort

This repository contains a two-stage-tracker. The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to a Deep Sort algor

Mike 3.6k Jan 05, 2023
A Python Library for Graph Outlier Detection (Anomaly Detection)

PyGOD is a Python library for graph outlier detection (anomaly detection). This exciting yet challenging field has many key applications, e.g., detect

PyGOD Team 757 Jan 04, 2023
A command line simple note taking app

Why yet another note taking program? note was designed with a very specific target in mind: me, and my 2354 scraps of paper. It runs from the command

64 Nov 20, 2022