QueryInst: Parallelly Supervised Mask Query for Instance Segmentation

• TL;DR: QueryInst is a simple and effective query based instance segmentation method driven by parallel supervision on dynamic mask heads, which outperforms previous arts in terms of both accuracy and speed.

QueryInst: Parallelly Supervised Mask Query for Instance Segmentation,

by Yuxin Fang*, Shusheng Yang*, Xinggang Wang†, Yu Li, Chen Fang, Ying Shan, Bin Feng, Wenyu Liu.

(*) equal contribution, (†) corresponding author.

arXiv technical report (arXiv 2105.01928)

• This repo serves as the official implementation for QueryInst, based on mmdetection and built upon Sparse R-CNN & DETR. Implantations based on Detectron2 will be released in the near future.

• This project is under active development, we will extend QueryInst to a wide range of instance-level recognition tasks.

Updates

[06/05/2021] 🌟 QueryInst training and inference code has been released!

Getting Started

• Our project is mainly developed on mmdetection toolbox (931d96), please refer to the mmdetection official installation.
• Install QueryInst by:

python setup.py develop

• Prepare datasets:

mkdir data && cd data
ln -s /path/to/coco coco

• Training QueryInst with single GPU:

python tools/train.py configs/queryinst/queryinst_r50_fpn_1x_coco.py

• Training QueryInst with multi GPUs:

./tools/dist_train.sh configs/queryinst/queryinst_r50_fpn_1x_coco.py 8

• Test QueryInst on COCO val set with single GPU:

python tools/test.py configs/queryinst/queryinst_r50_fpn_1x_coco.py PATH/TO/CKPT.pth --eval bbox segm

• Test QueryInst on COCO val set with multi GPUs:

./tools/dist_test.sh configs/queryinst/queryinst_r50_fpn_1x_coco.py PATH/TO/CKPT.pth 8 --eval bbox segm

Main Results on COCO val

Citation

If you find our paper and code useful in your research, please consider giving a star ⭐ and citation ?? :

@article{QueryInst,
  title={QueryInst: Parallelly Supervised Mask Query for Instance Segmentation},
  author={Fang, Yuxin and Yang, Shusheng and Wang, Xinggang and Li, Yu and Fang, Chen and Shan, Ying and Feng, Bin and Liu, Wenyu},
  journal={arXiv preprint arXiv:2105.01928},
  year={2021}
}

TODO

• QueryInst training and inference code.
• QueryInst based on Detectron2 toolbox will be released in the near future.
• QueryInst configurations for Cityscapes and YouTube-VIS.
• QueryInst pretrain weights.