Track to Detect and Segment: An Online Multi-Object Tracker (CVPR 2021)

Overview

Track to Detect and Segment: An Online Multi-Object Tracker (CVPR 2021)

Track to Detect and Segment: An Online Multi-Object Tracker
Jialian Wu, Jiale Cao, Liangchen Song, Yu Wang, Ming Yang, Junsong Yuan
In CVPR, 2021. [Paper] [Project Page] [Demo (YouTube)]

Many thanks to CenterTrack authors for their great framework!

Installation

Please refer to INSTALL.md for installation instructions.

Run Demo

We reuse the demo script from CenterTrack. Before run the demo, first download our trained models: CrowdHuman model (2D tracking), MOT model (2D tracking) or nuScenes model (3D tracking). Then, put the models in TraDeS_ROOT/models/ and cd TraDeS_ROOT/src/. The demo result will be saved as a video in TraDeS_ROOT/results/.

2D Tracking Demo

Demo for a video clip from MOT dataset: Run the demo (using the MOT model):

python demo.py tracking --dataset mot --load_model ../models/mot_half.pth --demo ../videos/mot_mini.mp4 --pre_hm --ltrb_amodal --pre_thresh 0.5 --track_thresh 0.4 --inference --clip_len 3 --trades --save_video --resize_video --input_h 544 --input_w 960

Demo for a video clip which we randomly selected from YouTube: Run the demo (using the CrowdHuman model):

python demo.py tracking --load_model ../models/crowdhuman.pth --num_class 1 --demo ../videos/street_2d.mp4 --pre_hm --ltrb_amodal --pre_thresh 0.5 --track_thresh 0.5 --inference --clip_len 2 --trades --save_video --resize_video --input_h 480 --input_w 864

Demo for your own video or image folder: Please specify the file path after --demo and run (using the CrowdHuman model):

python demo.py tracking --load_model ../models/crowdhuman.pth --num_class 1 --demo $path to your video or image folder$ --pre_hm --ltrb_amodal --pre_thresh 0.5 --track_thresh 0.5 --inference --clip_len 2 --trades --save_video --resize_video --input_h $your_input_h$ --input_w $your_input_w$

(Some Notes: (i) For 2D tracking, the models are only used for person tracking, since our method is only trained on CrowdHuman or MOT. You may train a model on COCO or your own dataset for multi-category 2D object tracking. (ii) --clip_len is set to 3 for MOT; otherwise, it should be 2. You may refer to our paper for this detail. (iii) The CrowdHuman model is more able to generalize to real world scenes than the MOT model. Note that both datasets are in non-commercial licenses. (iii) input_h and input_w shall be evenly divided by 32.)

3D Tracking Demo

Demo for a video clip from nuScenes dataset: Run the demo (using the nuScenes model):

python demo.py tracking,ddd --dataset nuscenes --load_model ../models/nuscenes.pth --demo ../videos/nuscenes_mini.mp4 --pre_hm --track_thresh 0.1 --inference --clip_len 2 --trades --save_video --resize_video --input_h 448 --input_w 800 --test_focal_length 633

(You will need to specify test_focal_length for monocular 3D tracking demo to convert the image coordinate system back to 3D. The value 633 is half of a typical focal length (~1266) in nuScenes dataset in input resolution 1600x900. The mini demo video is in an input resolution of 800x448, so we need to use a half focal length. You don't need to set the test_focal_length when testing on the original nuScenes data.)

You can also refer to CenterTrack for the usage of webcam demo (code is available in this repo, but we have not tested yet).

Benchmark Evaluation and Training

Please refer to Data.md for dataset preparation.

2D Object Tracking

MOT17 Val MOTA↑ IDF1↑ IDS↓
Our Baseline 64.8 59.5 1055
CenterTrack 66.1 64.2 528
TraDeS (ours) 68.2 71.7 285

Test on MOT17 validation set: Place the MOT model in $TraDeS_ROOT/models/ and run:

sh experiments/mot17_test.sh

Train on MOT17 halftrain set: Place the pretrained model in $TraDeS_ROOT/models/ and run:

sh experiments/mot17_train.sh

3D Object Tracking

nuScenes Val AMOTA↑ AMOTP↓ IDSA↓
Our Baseline 4.3 1.65 1792
CenterTrack 6.8 1.54 813
TraDeS (ours) 11.8 1.48 699

Test on nuScenes validation set: Place the nuScenes model in $TraDeS_ROOT/models/. You need to change the MOT and nuScenes dataset API versions due to their conflicts. The default installed versions are for MOT dataset. For experiments on nuScenes dataset, please run:

sh nuscenes_switch_version.sh

sh experiments/nuScenes_test.sh

To switch back to the API versions for MOT experiments, you can run:

sh mot_switch_version.sh

Train on nuScenes train set: Place the pretrained model in $TraDeS_ROOT/models/ and run:

sh experiments/nuScenes_train.sh

Train on Static Images

We follow CenterTrack which uses CrowdHuman to pretrain 2D object tracking model. Only the training set is used.

sh experiments/crowdhuman.sh

The trained model is available at CrowdHuman model.

Instance Segmentation Tracking

Code will be released later on after we clean it up. Our implementation is based on here.

Citation

If you find it useful in your research, please consider citing our paper as follows:

@inproceedings{Wu2021TraDeS,
title={Track to Detect and Segment: An Online Multi-Object Tracker},
author={Wu, Jialian and Cao, Jiale and Song, Liangchen and Wang, Yu and Yang, Ming and Yuan, Junsong},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}}
Owner
Jialian Wu
Ph.D. Candidate at SUNY Buffalo
Jialian Wu
Bot made by BLACKSTORM[BM] Contact Us - t.me/BLACKSTORM18

ᴡʜᴀᴛ ɪs ᴊᴀʀᴠɪs sᴇᴄᴜʀɪᴛʏ ʙᴏᴛ ᴊᴀʀᴠɪs ʙᴏᴛ ɪs ᴛᴇʟᴇɢʀᴀᴍ ɢʀᴏᴜᴘ ᴍᴀɴᴀɢᴇʀ ʙᴏᴛ ᴡɪᴛʜ ᴍᴀɴʏ ғᴇᴀᴛᴜʀᴇs. ᴛʜɪs ʙᴏᴛ ʜᴇʟᴘs ʏᴏᴜ ᴛᴏ ᴍᴀɴᴀɢᴇ ʏᴏᴜʀ ɢʀᴏᴜᴘs ᴇᴀsɪʟʏ. ᴏʀɪɢɪɴᴀʟʟʏ ᴀ

1 Dec 11, 2021
Discord bot that displays Jazz Jackrabbit 2 server status, current gamemode as "Playing.." status

JJ2-server-status-discord-bot Discord bot that displays Jazz Jackrabbit 2 server status, current gamemode as "Playing.." status How to setup: 0. Downl

2 Dec 09, 2021
Python 3 SDK/Wrapper for Huobi Crypto Exchange Api

This packages intents to be an idiomatic PythonApi wrapper for https://www.huobi.com/ Huobi Api Doc: https://huobiapi.github.io/docs Showcase TODO Con

3 Jul 28, 2022
WhatsApp Api Python - This documentation aims to exemplify the use of Moorse Whatsapp API in Python

WhatsApp API Python ChatBot Este repositório contém uma aplicação que se utiliza

Moorse.io 3 Jan 08, 2022
vk Bot because of which everyone will lag

VK-crash-bot open cmd and write: "pip install vk-api" To configure the bot, you need to open main.py and set the value to such variables as "token" an

NotQuki 0 Jun 05, 2022
Best Buy purchase bot

B3 Best-Buy-Bot. Written in Python NOTICE: Don't be a disgrace to society. Don't use this for any mass buying/reselling purposes. About B3 is a bot th

Dogey11 8 Aug 15, 2022
Automate TikTok follower bot, like bot, share bot, view bot and more using selenium

Zefoy TikTok Automator Automate TikTok follower bot, like bot, share bot, view bot and more using selenium. Click here to report bugs. Usage Download

555 Dec 30, 2022
Template to create a telegram bot in python

Template for Telegram Bot Template to create a telegram bot in python. How to Run First add src to PYTHONPATH: export PYTHONPATH=${PWD} Then run: pyt

Ali Hejazizo 12 Dec 24, 2022
Um bot para contar quantas vezes o meu amigo troca de pfp/nick/tag essas coisas ae pq aquele mlk n para quieto

EkiBot Um bot que tem apenas as suas funções de audit log com as PFP's (avatares) dos usuários Pode ser usado para um usuário em específico, ou até me

Samuel 3 Aug 11, 2021
UP It is a script to notify of a new update of your project, done in python and using GitHub, to modify the versions to notify users.

UP-Updater UP It is a script to notify of a new update of your project, done in python and using GitHub, to modify the versions to notify users. Requi

Made in 4 Oct 28, 2021
Senditapp.com bot spammer, spam your friends

Sendit Spammer Python ⚠️ I am not responsible for how you use this tool. This tool is against "Sendit" ToS and shall not be used in a production envir

Glaukio 1 Dec 31, 2021
Osmnx-examples - Usage examples, demos, and tutorials for OSMnx.

OSMnx Examples OSMnx is a Python package to work with street networks and other spatial data from OpenStreetMap: retrieve, model, analyze, and visuali

Geoff Boeing 1.2k Jan 03, 2023
Example-bot-discord - Example bot discord xD

example-python-bot-discord Clone this repository Grab a token on Discord's devel

Amitminer 1 Mar 14, 2022
Evernote SDK for Python

Evernote SDK for Python Evernote API version 1.28 This SDK is intended for use with Python 2.X For Evernote's beta Python 3 SDK see https://github.com

Evernote 612 Dec 30, 2022
Demonstrating attacks, mitigations, and monitoring on AWS

About Inspectaroo is a web app which allows users to upload images to view metadata. It is designed to show off many AWS services including EC2, Lambd

Alex McCormack 1 Feb 11, 2022
Bavera is an extensive and extendable Python 3.x library for the Discord API

Bavera is an extensive and extendable Python 3.x library for the Discord API. Bavera boasts the following major features: Expressive, functiona

Bavera 1 Nov 17, 2021
Petpy is an easy-to-use and convenient Python wrapper for the Petfinder API.

Petpy is an easy-to-use and convenient Python wrapper for the Petfinder API. Includes methods for parsing output JSON into pandas DataFrames for easier data analysis

Aaron Schlegel 27 Nov 19, 2022
Unofficial GoPro API Library for Python - connect to GoPro via WiFi.

GoPro API for Python Unofficial GoPro API Library for Python - connect to GoPro cameras via WiFi. Compatibility: HERO3 HERO3+ HERO4 (including HERO Se

Konrad Iturbe 1.3k Jan 01, 2023
Hydro Quebec API wrapper.

HydroQC Hydro Quebec API wrapper. This is a package to access some functionalities of Hydro Quebec API that are not documented. Documentation https://

Olivier BEAU 9 Dec 02, 2022
Python client and module for BGP Ranking

Python client and module for BGP Ranking THis project will make querying BGP Ranking easier. Installation pip install pybgpranking Usage Command line

D4 project 3 Dec 16, 2021