Source codes of CenterTrack++ in 2021 ICME Workshop on Big Surveillance Data Processing and Analysis

Overview

MOT Tracked object bounding box association (CenterTrack++)

New association method based on CenterTrack. Two new branches (Tracked Size and IOU) are added onto the original CenterTrack tracker. The proposed method enables the computation of IOU distance matrix for more accurate object association compared to single displacement offset in the original CenterTrack.

Modification to CenterTrack method, image modified from CenterTrack

Abstract

The recent development of multi-object tracking (MOT) on point-based joint detection and tracking methods has attracted much research attention. CenterTrack tracking algorithm is one of such promising methods. It achieves state-of-the-art tracking performance using a simple detection model and single-frame spatial offsets to localize objects and predict their associations in a single network. However, this method still suffers from high identity switches due to the inferior association method. Only point displacement distance matrix is used to associate objects, which is not robust to deal with occlusion scenarios. To reduce the high number of identity switches and improve the tracking accuracy, more effective spatial information should be used in association. In this paper, we propose to incorporate a simple tracked object bounding box and overlapping prediction based on the current frame onto the CenterTrack algorithm. Specifically, we propose a Intersection over Union (IOU) distance cost matrix in the association step instead of point displacement distance. We evaluate our proposed tracker on the MOT17 test dataset, showing that our proposed method can reduce identity switches significantly by 22.6% and obtain a notable improvement of 1.5% in IDF1 compared to the original CenterTrack’s under the same tracklet lifetime.

Main Contributions

  • Proposed two branches (tracked box size and IOU)on top of the existing CenterTrack method for IOU distance metric computation in object association
  • Evaluation the proposed method on MOT17 dataset and obtain significant reduction in IDs and notable improvements in tracking accuracy score

Two new branches

The idea of the proposed method is to enhance the original displacement only association. Inspired by the IOU distance in SORT and IOU-Tracker, IOU distance can be used for more accurate object association across frames. IOU distance is calculated as 1 - IOU(bounding box of detected object in the previous frame and the predicted tracked object bounding box in the previous frame based on the current frame)

Tracked Object Size prediction

In order to obtain the IOU distance, the bounding box of the tracked object in the previous frame should be learnt. In this project, two methods were used to learn the tracked bounding box.

Tracking_wh: Directly learn the width and height of the tracked object bounding box in the previous frame.

Tracking_ltrb: Learn the offsets of the left, top, right and bottom of bounding box from the tracked object center in the previous frame.

The tracking_wh(left) and tracking_ltrb(right) approach illustration.

IOU prediction

To further suppress inaccurate association, the IOU value of the tracked object bounding box in adjacent frames is learnt to provide a threshold to filter unlikely associations. We would set the IOU distance to infinity if IOU distance > IOU.

Association Method

Main results

Comparison with other SOTA tracker on MOT17 test set

Note: S= Spatial features, A=appearance features

Tracker Association Features MOTA IDF1 IDs
TubeTK S 63 58.6 4137
CenterTrack S 67.8 64.7 3039
Ours S 68.1 66.2 2352
SST A 52.4 49.5 8431
CTrackerV1 S+A 66.6 57.4 5529
DEFT S+A 66.6 65.4 2823
FairMOT S+A 73.7 72.3 3303

Ablative studies on tracked size prediction method

Tracking_wh

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 71.1 66.7 204 3.6 29.3
Combined 70.9 66.2 233 3.9 29.6
DIS→IOU 70 66.2 218 3.9 29.5
IOU→DIS 69.8 66.8 185 3.6 29.2

Tracking_ltrb

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 72.4 66.7 191 3.8 29.2
Combined 70.8 66.5 236 3.8 29.3
DIS→IOU 70.5 66.6 202 3.8 29.2
IOU→DIS 71.4 66.7 166 3.8 29.2

Installation

Please refer to INSTALL.md for installation instructions.

Training and Evaluation

  • Download the crowdhuman pretrained model from xinyizhou/CenterTrack MODEL ZOO.md to models
  • prepare the data and convert it into COCO format refer to the original CenterTrack repo.
  • change the dataset root directory data_dir in opt.py
  • ablative studies for tracking_wh and tracking_ltrb approach respectively with five association method (IOU,DIS,Combined, IOU→DIS, DIS→IOU)
sh experiments/mot17val_tracking_wh.sh

sh experiments/mot17val_tracking_ltrb.sh

The trained model on MOT17val dataset using two approach are available in google drive, tracking_ltrb_70val.pth, tracking_wh_70val.pth.

  • Train on full mot17 training set and run model on the test set for evaluation
sh experiments/mot17full.sh

The trained models on full MOT17 dataset using ltrb approach is available in the google drive.

Demo comparison

Occlusion case

Original CenterTrack (left) vs CenterTrack++ (right)

Object exiting the frame

Original CenterTrack (left) vs CenterTrack++ (right)

Acknowledgement

A large part of the code is adapted from xingyizhou/CenterTrack, thanks for their wonderful inspiration.

Citation

If you find this paper and code useful in your research, please cite our papers.

@misc{yang2021multiobject,
      title={Multi-object Tracking with Tracked Object Bounding Box Association}, 
      author={Nanyang Yang and Yi Wang and Lap-Pui Chau},
      year={2021},
      eprint={2105.07901},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
Owner
Nanyang Technological University Information Engineering and Media Student
Proof of concept GnuCash Webinterface

Proof of Concept GnuCash Webinterface This may one day be a something truly great. Milestones [ ] Browse accounts and view transactions [ ] Record sim

Josh 14 Dec 28, 2022
make ASCII Art by Deep Learning

DeepAA This is convolutional neural networks generating ASCII art. This repository is under construction. This work is accepted by NIPS 2017 Workshop,

OsciiArt 1.4k Dec 28, 2022
Intelligent Video Analytics toolkit based on different inference backends.

English | 中文 OpenIVA OpenIVA is an end-to-end intelligent video analytics development toolkit based on different inference backends, designed to help

Quantum Liu 15 Oct 27, 2022
EfficientDet (Scalable and Efficient Object Detection) implementation in Keras and Tensorflow

EfficientDet This is an implementation of EfficientDet for object detection on Keras and Tensorflow. The project is based on the official implementati

1.3k Dec 19, 2022
The implemetation of Dynamic Nerual Garments proposed in Siggraph Asia 2021

DynamicNeuralGarments Introduction This repository contains the implemetation of Dynamic Nerual Garments proposed in Siggraph Asia 2021. ./GarmentMoti

42 Dec 27, 2022
Mining-the-Social-Web-3rd-Edition - The official online compendium for Mining the Social Web, 3rd Edition (O'Reilly, 2018)

Mining the Social Web, 3rd Edition The official code repository for Mining the Social Web, 3rd Edition (O'Reilly, 2019). The book is available from Am

Mikhail Klassen 838 Jan 01, 2023
A Distributional Approach To Controlled Text Generation

A Distributional Approach To Controlled Text Generation This is the repository code for the ICLR 2021 paper "A Distributional Approach to Controlled T

NAVER 102 Jan 07, 2023
Code for "Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans" CVPR 2021 best paper candidate

News 05/17/2021 To make the comparison on ZJU-MoCap easier, we save quantitative and qualitative results of other methods at here, including Neural Vo

ZJU3DV 748 Jan 07, 2023
Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution

unfoldedVBA Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution This repository contains the Pytorch implementation of the unrolled

Yunshi HUANG 2 Jul 10, 2022
ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representa

Bats Research 94 Nov 21, 2022
ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection

ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection This repository contains implementation of the

Visual Understanding Lab @ Samsung AI Center Moscow 190 Dec 30, 2022
A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers.

Dying Light 2 PAKFile Utility A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers. This tool aims to make PAKFile (.pak files) modding a

RHQ Online 12 Aug 26, 2022
Original Pytorch Implementation of FLAME: Facial Landmark Heatmap Activated Multimodal Gaze Estimation

FLAME Original Pytorch Implementation of FLAME: Facial Landmark Heatmap Activated Multimodal Gaze Estimation, accepted at the 17th IEEE Internation Co

Neelabh Sinha 19 Dec 17, 2022
Multi-Horizon-Forecasting-for-Limit-Order-Books

Multi-Horizon-Forecasting-for-Limit-Order-Books This jupyter notebook is used to demonstrate our work, Multi-Horizon Forecasting for Limit Order Books

Zihao Zhang 116 Dec 23, 2022
QuanTaichi evaluation suite

QuanTaichi: A Compiler for Quantized Simulations (SIGGRAPH 2021) Yuanming Hu, Jiafeng Liu, Xuanda Yang, Mingkuan Xu, Ye Kuang, Weiwei Xu, Qiang Dai, W

Taichi Developers 120 Jan 04, 2023
Implementation of "Glancing Transformer for Non-Autoregressive Neural Machine Translation"

GLAT Implementation for the ACL2021 paper "Glancing Transformer for Non-Autoregressive Neural Machine Translation" Requirements Python = 3.7 Pytorch

117 Jan 09, 2023
PiRapGenerator - Make anyone rap the digits of pi

PiRapGenerator Make anyone rap the digits of pi (sample files are of Ted Nivison

7 Oct 02, 2022
Code for CVPR 2021 paper: Anchor-Free Person Search

Introduction This is the implementationn for Anchor-Free Person Search in CVPR2021 License This project is released under the Apache 2.0 license. Inst

158 Jan 04, 2023
PyTorch implementation of EfficientNetV2

[NEW!] Check out our latest work involution accepted to CVPR'21 that introduces a new neural operator, other than convolution and self-attention. PyTo

Duo Li 375 Jan 03, 2023
Code implementation of "Sparsity Probe: Analysis tool for Deep Learning Models"

Sparsity Probe: Analysis tool for Deep Learning Models This repository is a limited implementation of Sparsity Probe: Analysis tool for Deep Learning

3 Jun 09, 2021