Official implementation of "OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association" in PyTorch.

Last update: Dec 29, 2022

Related tags

Deep Learning openpifpaf

Overview

openpifpaf

Continuously tested on Linux, MacOS and Windows:
New 2021 paper:

OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association
Sven Kreiss, Lorenzo Bertoni, Alexandre Alahi, 2021.

Many image-based perception tasks can be formulated as detecting, associating and tracking semantic keypoints, e.g., human body pose estimation and tracking. In this work, we present a general framework that jointly detects and forms spatio-temporal keypoint associations in a single stage, making this the first real-time pose detection and tracking algorithm. We present a generic neural network architecture that uses Composite Fields to detect and construct a spatio-temporal pose which is a single, connected graph whose nodes are the semantic keypoints (e.g., a person's body joints) in multiple frames. For the temporal associations, we introduce the Temporal Composite Association Field (TCAF) which requires an extended network architecture and training method beyond previous Composite Fields. Our experiments show competitive accuracy while being an order of magnitude faster on multiple publicly available datasets such as COCO, CrowdPose and the PoseTrack 2017 and 2018 datasets. We also show that our method generalizes to any class of semantic keypoints such as car and animal parts to provide a holistic perception framework that is well suited for urban mobility such as self-driving cars and delivery robots.

Previous CVPR 2019 paper.

Guide

Detailed documentation is in our OpenPifPaf Guide. For developers, there is also the DEV Guide which is the same guide but based on the latest code in the main branch.

Examples

Image credit: "Learning to surf" by fotologic which is licensed under CC-BY-2.0.
Created with:

pip3 install openpifpaf matplotlib
python3 -m openpifpaf.predict docs/coco/000000081988.jpg --image-output

Here is the tutorial for body, foot, face and hand keypoints. Example:

Image credit: Photo by Lokomotive74 which is licensed under CC-BY-4.0.
Created with:

python -m openpifpaf.predict guide/wholebody/soccer.jpeg \
  --checkpoint=shufflenetv2k30-wholebody --line-width=2 --image-output

Here is the tutorial for car keypoints. Example:

Image credit: Photo by Ninaras which is licensed under CC-BY-SA 4.0.

Created with:

python -m openpifpaf.predict guide/images/peterbourg.jpg \
  --checkpoint shufflenetv2k16-apollo-24 -o images \
  --instance-threshold 0.05 --seed-threshold 0.05 \
  --line-width 4 --font-size 0

Here is the tutorial for animal keypoints (dogs, cats, sheep, horses and cows). Example:

python -m openpifpaf.predict guide/images tappo_loomo.jpg \
  --checkpoint=shufflenetv2k30-animalpose \
  --line-width=6 --font-size=6 --white-overlay=0.3 \
  --long-edge=500

Commercial License

This software is available for licensing via the EPFL Technology Transfer Office (https://tto.epfl.ch/, [email protected]).

Comments

when i try to run v0.14.0 from source code, it raise ImportError.

I clone the source code from github, then i want to run the openpifpaf/src/openpifpaf/predict.py file.And it raise error.I tracked down the issue and found that the error came from cpp_extension.register_ops()。

extfinder = importlib.machinery.FileFinder(lib_dir, loader_details) ext_specs = extfinder.find_spec("_cpp") if ext_specs is None: raise ImportError

It needs some openpifpaf._cpp files where i found in setup.py. I want to run from the source code, and i do not know how to build this openpifpaf._cpp file,please help!!

opened by KingArtherTT 1
Openpifpaf MPII PlugIn

Train an openpifpaf model with the MPII dataset. Same code as a plugin and reference can be found here: https://github.com/DuncanZauss/openpifpaf_mpii

opened by DuncanZauss 0

Releases(v0.14.0)

v0.14.0(Feb 14, 2022)
create openpifpaf-vita package, with separate development from openpifpaf

deploy openpifpaf-vita guide

Source code(tar.gz)
Source code(zip)

This is the official PyTorch implementation of the paper "TransFG: A Transformer Architecture for Fine-grained Recognition" (Ju He, Jie-Neng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai, Changhu Wang, Alan Yuille).

TransFG: A Transformer Architecture for Fine-grained Recognition Official PyTorch code for the paper: TransFG: A Transformer Architecture for Fine-gra

307 Jan 3, 2023

StyleGAN2-ADA - Official PyTorch implementation

Need Help? If you’re new to StyleGAN2-ADA and looking to get started, please check out this video series from a course Lia Coleman and I taught in Oct

217 Jan 4, 2023

Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

123 Dec 27, 2022

Official PyTorch implementation of RobustNet (CVPR 2021 Oral)

RobustNet (CVPR 2021 Oral): Official Project Webpage Codes and pretrained models will be released soon. This repository provides the official PyTorch

173 Dec 21, 2022

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

PyTorch Implementation of Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers 1 Using Colab Please notic

489 Jan 7, 2023

[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

153 Dec 14, 2022

Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Introduction Pytorch implementation of Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Expert. | paper Song Park1

97 Dec 23, 2022

Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

71 Nov 18, 2022

Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Hypercorrelation Squeeze for Few-Shot Segmentation This is the implementation of the paper "Hypercorrelation Squeeze for Few-Shot Segmentation" by Juh

165 Dec 28, 2022

Official implementation of "OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association" in PyTorch.

Related tags

Overview

openpifpaf

Guide

Examples

Commercial License

You might also like...

This is the official PyTorch implementation of the paper "TransFG: A Transformer Architecture for Fine-grained Recognition" (Ju He, Jie-Neng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai, Changhu Wang, Alan Yuille).

StyleGAN2-ADA - Official PyTorch implementation

Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

Official PyTorch implementation of RobustNet (CVPR 2021 Oral)

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Comments

when i try to run v0.14.0 from source code, it raise ImportError.

Openpifpaf MPII PlugIn

Releases(v0.14.0)

v0.14.0(Feb 14, 2022)

Owner

VITA lab at EPFL

Video Autoencoder: self-supervised disentanglement of 3D structure and motion

Official code repository for ICCV 2021 paper: Gravity-Aware Monocular 3D Human Object Reconstruction

Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption

Acoustic mosquito detection code with Bayesian Neural Networks

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Scripts and outputs related to the paper Prediction of Adverse Biological Effects of Chemicals Using Knowledge Graph Embeddings.

This repo implements a 3D segmentation task for an airport baggage dataset.

Neuron Merging: Compensating for Pruned Neurons (NeurIPS 2020)

[ECE NTUA] 👁 Computer Vision - Lab Projects & Theoretical Problem Sets (2020-2021)

一个多模态内容理解算法框架，其中包含数据处理、预训练模型、常见模型以及模型加速等模块。

Omniscient Video Super-Resolution

SeMask: Semantically Masked Transformers for Semantic Segmentation.

A supplementary code for Editable Neural Networks, an ICLR 2020 submission.

Source codes for "Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs"

NovelD: A Simple yet Effective Exploration Criterion

Asynchronous Advantage Actor-Critic in PyTorch

Notepy is a full-featured Notepad Python app

[ICLR 2021 Spotlight Oral] "Undistillable: Making A Nasty Teacher That CANNOT teach students", Haoyu Ma, Tianlong Chen, Ting-Kuei Hu, Chenyu You, Xiaohui Xie, Zhangyang Wang

An off-line judger supporting distributed problem repositories

Contrastive Learning of Structured World Models