UOTA: Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration
This repository is the official PyTorch implementation of UOTA (Unsupervised OuTlier Arbitration).
0 Requirements
- Python 3.6
- PyTorch install = 1.6.0
- torchvision install = 0.7.0
- CUDA 10.1
- Apex with CUDA extension
- Other dependencies: opencv-python, scipy, pandas, numpy
1 Pretraining
We release a demo to pretrain ResNet50 on ImageNet1K with SwAV+UOTA pretrained models.
1.1 SwAV+UOTA pretrain
To train SwAV+UOTA on a single node with 4 gpus for 200 epochs, run:
DATASET_PATH="path/to/ImageNet1K/train"
EXPERIMENT_PATH="path/to/experiment"
python -m torch.distributed.launch --nproc_per_node=4 main_uota.py \
--data_path ${DATASET_PATH} \
--nmb_crops 2 6 \
--size_crops 224 96 \
--min_scale_crops 0.14 0.05 \
--max_scale_crops 1. 0.14 \
--crops_for_assign 0 1 \
--use_pil_blur true \
--epochs 200 \
--warmup_epochs 0 \
--batch_size 64 \
--base_lr 0.6 \
--final_lr 0.0006 \
--uota_tau 350. \
--epoch_uota_starts 100 \
--wd 0.000001 \
--use_fp16 true \
--dist_url "tcp://localhost:40000" \
--arch uota_r50 \
--sync_bn pytorch \
--dump_path ${EXPERIMENT_PATH}
2 Linear Evaluation
To train a linear classifier on frozen features out of deep network pretrained via various self-supervised pretraining methods, run:
DATASET_PATH="path/to/ImageNet1K"
EXPERIMENT_PATH="path/to/experiment"
LINCLS_PATH="path/to/lincls"
python -m torch.distributed.launch --nproc_per_node=4 eval_linear.py \
--data_path ${DATASET_PATH} \
--arch resnet50 \
--lr 1.2 \
--dump_path ${LINCLS_PATH} \
--pretrained ${EXPERIMENT_PATH}/swav_uota_r50_e200_pretrained.pth \
--batch_size 64 \
--num_classes 100 \
3 Results
To compare with SwAV fairly, we provide a SwAV+UOTA model with ResNet-50 architecture pretrained on ImageNet1K for 200 epochs, and release the pretrained model and the linear classier.
| method | epochs | batch-size | multi-crop | ImageNet1K top-1 acc. | pretrained model | linear classifier |
|---|---|---|---|---|---|---|
| SwAV | 200 | 256 | 2x224 + 6x96 | 72.7 | / | / |
| SwAV + UOTA | 200 | 256 | 2x224 + 6x96 | 73.5 | pretrained | linear |
4 Citation
@InProceedings{wang2021NeurIPS,
title={Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration},
author={Wang, Yu and Lin, Jingyang and Zou, Jingjing and Pan, Yingwei and Yao, Ting and Mei, Tao},
booktitle={NeurIPS},
year={2021},
}
287 Jan 4, 2023
75 Nov 24, 2022
297 Dec 26, 2022
16 Nov 4, 2020
9 Nov 14, 2022
6.6k Jan 3, 2023
181 Dec 18, 2022
12 Jan 10, 2022
64 Nov 2, 2022
2 Jul 18, 2022
6 Dec 13, 2022
38 Dec 07, 2022
68 Nov 28, 2022
54 Dec 06, 2022
16 Nov 01, 2022
261 Jan 09, 2023
2.3k Jan 08, 2023
2.3k Jan 01, 2023
2 Jul 27, 2022
9 Mar 14, 2022
98 Nov 06, 2022
11 Dec 26, 2022
39 Sep 21, 2022
12 Oct 23, 2022
[email protected] Lab">
192 Jan 05, 2023
4 Aug 24, 2022
15 Jun 10, 2022
66 Dec 31, 2022
18 Dec 27, 2022