SoCo

[NeurIPS 2021 Spotlight] Aligning Pretraining for Detection via Object-Level Contrastive Learning

By Fangyun Wei*, Yue Gao*, Zhirong Wu, Han Hu, Stephen Lin.

* Equal contribution.

Introduction

Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning. Such generality for transfer learning, however, sacrifices specificity if we are interested in a certain downstream task. We argue that this could be sub-optimal and thus advocate a design principle which encourages alignment between the self-supervised pretext task and the downstream task. In this paper, we follow this principle with a pretraining method specifically designed for the task of object detection. We attain alignment in the following three aspects:

1. object-level representations are introduced via selective search bounding boxes as object proposals;
2. the pretraining network architecture incorporates the same dedicated modules used in the detection pipeline (e.g. FPN);
3. the pretraining is equipped with object detection properties such as object-level translation invariance and scale invariance. Our method, called Selective Object COntrastive learning (SoCo), achieves state-of-the-art results for transfer performance on COCO detection using a Mask R-CNN framework.

Architecture

Main results

The pretrained models will be available soon.

SoCo pre-trained models

Results on COCO with MaskRCNN R50-FPN

Results on COCO with MaskRCNN R50-C4

Get started

Requirements

The Dockerfile is included, please refer to it.

Prepare data with Selective Search

1. Generate Selective Search proposals

   python selective_search/generate_imagenet_ss_proposals.py

2. Filter out not valid proposals with filter strategy

   python selective_search/filter_ss_proposals_json.py

3. Post preprocessing for no proposals images

   python selective_search/filter_ss_proposals_json_post_no_prop.py

Pretrain with SoCo

Use SoCo FPN 100 epoch as example.

bash ./tools/SoCo_FPN_100ep.sh

Finetune detector

1. Copy the folder detectron2_configs to the root folder of Detectron2
2. Train the detectors with Detectron2

Citation

@article{wei2021aligning,
  title={Aligning Pretraining for Detection via Object-Level Contrastive Learning},
  author={Wei, Fangyun and Gao, Yue and Wu, Zhirong and Hu, Han and Lin, Stephen},
  journal={arXiv preprint arXiv:2106.02637},
  year={2021}
}