Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking

Last update: Dec 12, 2022

Related tags

Deep Learning ArTIST

Overview

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021)

Pytorch implementation of the ArTIST motion model. In this repo, there are

Training script for the Moving Agent network
Training script for the ArTIST motion model
Demo script for Inferring the likelihood of current observations (detections)
Demo script for Inpainting the missing observation/detections

Demo 1: Likelihood estimation of observation

Run:

python3 demo_scoring.py

This will generate the output in the temp/ar/log_p directory, look like this:

This demo gets as input a pretrained model of the Moving Agent Network (MA-Net), a pretrained model of ArTIST, the centroids (obtain centroids via the script in the utils), a demo test sample index and the number of clusters.

The model then evaluates the log-likelihood (lower the better) of all detections as the continuation of the observed sequence.

Demo 2: Sequence inpainting

Run:

python3 demo_inpainting.py

This will generate the multiple plauusible continuations of an observed motion, stored in the temp/ar/inpainting directory. One example looks like this:

For each generated future sequence, it computes the IoU between the last generated bounding box and the last groundtruth bounding box, as well as the mean IoU for the entire generated sequence and the groundtruth sequence.

Utilities

In this repo, there are a number of scripts to generate the required data to train/evaluate ArTIST.

prepare_data: Given the annotations of a dataset (e.g., MOT17), it extracts the motion sequences as well as the IDs of the social tracklets living the life span of the corresponding sequence, and stores it as a dictionary. If there are multiple tracking datasets that you wish to combine, you can use the merge_datasets() function inside this script.
clustering: Given the output dictionary of prepare_data script, this script performs the K-Means clustering and stores the centroids which are then used in the ArTIST model.
dataloader_ae and dataloader_ar: Given the post-processes version of the dataset dictionary (which can be done by running the post_process script), these two scripts define the dataloaders for training the MA-Net and ArTIST. Note that the dataloader of ArTIST uses the MA-Net to compute the social information. This can also be done jointly in an end-to-end fashion, which we observed almost no difference.
create_demo_test_subset: In order to run the demo scripts, you need to run this script. However, the demo test subset has been produced and stored in data/demo_test_subset.npy.

Data

You can download the required data from the Release and put it in data/ directory.

Citation

If you find this work useful in your own research, please consider citing:

@inproceedings{saleh2021probabilistic,
author={Saleh, Fatemeh and Aliakbarian, Sadegh and Rezatofighi, Hamid and Salzmann, Mathieu and Gould, Stephen},
title = {Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
year = {2021}
}

Multiple Object Tracking with Yolov5!

Tracking with yolov5 This implementation is for who need to tracking multi-object only with detector. You can easily track mult-object with your well

9 Nov 8, 2022

A New Approach to Overgenerating and Scoring Abstractive Summaries

We provide the source code for the paper "A New Approach to Overgenerating and Scoring Abstractive Summaries" accepted at NAACL'21. If you find the code useful, please cite the following paper.

4 Apr 3, 2022

Y. Zhang, Q. Yao, W. Dai, L. Chen. AutoSF: Searching Scoring Functions for Knowledge Graph Embedding. IEEE International Conference on Data Engineering (ICDE). 2020

AutoSF The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding" and this paper has been accepted by ICDE2020. News:

64 Dec 17, 2022

SiamMOT is a region-based Siamese Multi-Object Tracking network that detects and associates object instances simultaneously.

SiamMOT: Siamese Multi-Object Tracking

432 Dec 17, 2022

Comments

Re-creating paper results

Did you use implement the ArTIST paradigm in the SORT algorithm to achieve the results in your paper? If so, do you have an example of integrating the ArTIST motion model with SORT? I am trying to re-create the results of the paper.

How do I re-create the results you obtained in your paper?

opened by vineetrshenoy 1
dataloader.py: shape mismatch

when i use dataloader.py to load the data, here comes a error:could not broadcast input array from shape (2) into shape (4) in line 33 of dataloader.py, I wonder how to fix the bug and what is the data format in data/postp_combined_path_mot_train.npy, thanks for your help.

opened by guileihu 0

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking

Related tags

Overview

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021)

Demo 1: Likelihood estimation of observation

Demo 2: Sequence inpainting

Utilities

Data

Citation

You might also like...

Multiple Object Tracking with Yolov5!

A New Approach to Overgenerating and Scoring Abstractive Summaries

Y. Zhang, Q. Yao, W. Dai, L. Chen. AutoSF: Searching Scoring Functions for Knowledge Graph Embedding. IEEE International Conference on Data Engineering (ICDE). 2020

The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding"

Image-popularity-score - A novel deep regression method for image scoring.

Object tracking and object detection is applied to track golf puts in real time and display stats/games.

Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

Object Detection and Multi-Object Tracking

SiamMOT is a region-based Siamese Multi-Object Tracking network that detects and associates object instances simultaneously.

Comments

Re-creating paper results

dataloader.py: shape mismatch

Releases(data-release)

data-release(May 9, 2021)

Owner

Fatemeh

Geometric Deep Learning Extension Library for PyTorch

A PyTorch Implementation of Gated Graph Sequence Neural Networks (GGNN)

This code is for our paper "VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers"

《K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters》(2020)

LabelImg is a graphical image annotation tool.

Heart Arrhythmia Classification

[ICLR 2022] Contact Points Discovery for Soft-Body Manipulations with Differentiable Physics

Unsupervised Image to Image Translation with Generative Adversarial Networks

Waymo motion prediction challenge 2021: 3rd place solution

Implementation of ReSeg using PyTorch

Code to reproduce experiments in the paper "Explainability Requires Interactivity".

A Pytorch implementation of "LegoNet: Efficient Convolutional Neural Networks with Lego Filters" (ICML 2019).

This is the repo of the manuscript "Dual-branch Attention-In-Attention Transformer for speech enhancement"

CATE: Computation-aware Neural Architecture Encoding with Transformers

Training, generation, and analysis code for Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics

Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals.

Code for "Adversarial Training for a Hybrid Approach to Aspect-Based Sentiment Analysis

meProp: Sparsified Back Propagation for Accelerated Deep Learning (ICML 2017)

Multi-tool reverse engineering collaboration solution.

Production First and Production Ready End-to-End Speech Recognition Toolkit