This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

Last update: Sep 18, 2022

Related tags

Deep Learning JigsawClustering

Overview

JigsawClustering

Jigsaw Clustering for Unsupervised Visual Representation Learning

Pengguang Chen, Shu Liu, Jiaya Jia

Introduction

This project provides an implementation for the CVPR 2021 paper "Jigsaw Clustering for Unsupervised Visual Representation Learning"

Installation

Environment

We verify our code on

4x2080Ti GPUs
CUDA 10.1
python 3.7
torch 1.6.0
torchvision 0.7.0

Other similar envirouments should also work properly.

Install

We use the SyncBN from apex, please install apex refer to https://github.com/NVIDIA/apex (SyncBN from pytorch should also work properly, we will verify it later.)

We use detectron2 for the training of detection tasks. If you are willing to finetune our pretrained model on the detection task, please install detectron2 refer to https://github.com/facebookresearch/detectron2

git clone https://github.com/Jia-Research-Lab/JigsawClustering.git
cd JigsawClustering/
pip install diffdist

Dataset

Please put the data under ./datasets. The directory looks like:

datasets
│
│───ImageNet/
│   │───class1/
│   │───class2/
│   │   ...
│   └───class1000/
│   
│───coco/
│   │───annotations/
│   │───train2017/
│   └───val2017/
│
│───VOC2012/
│   
└───VOC2007/

Results and pretrained model

The pretrained model is available at here.

Task	Dataset	Results
Linear Evaluation	ImageNet	66.4
Semi-Supervised 1%	ImageNet	40.7
Semi-Supervised 10%	ImageNet	63.0
Detection	COCO	39.3

Training

Pre-training on ImageNet

python main.py --dist-url 'tcp://localhost:10107' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --lr 0.03 --batch-size 256 --epoch 200 \
    --save-dir outputs/jigclu_pretrain/ \
    --resume outputs/jigclu_pretrain/model_best.pth.tar \
    --loss-t 0.3 \
    --cross-ratio 0.3 \
    datasets/ImageNet/

Linear evaluation on ImageNet

python main_lincls.py --dist-url 'tcp://localhost:10007' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --lr 10.0 --batch-size 256 \
    --prefix module.encoder. \
    --pretrained outputs/jigclu_pretrain/model_best.pth.tar \
    --save-dir outputs/jigclu_linear/ \
    datasets/ImageNet/

Semi-Supervised finetune on ImageNet

10% label

python main_semi.py --dist-url 'tcp://localhost:10102' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --batch-size 256 \
    --wd 0.0 --lr 0.01 --lr-last-layer 0.2 \
    --syncbn \
    --prefix module.encoder. \
    --labels-perc 10 \
    --pretrained outputs/jigclu_pretrain/model_best.pth.tar \
    --save-dir outputs/jigclu_semi_10p/ \
    datasets/ImageNet/

1% label

python main_semi.py --dist-url 'tcp://localhost:10101' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --batch-size 256 \
    --wd 0.0 --lr 0.02 --lr-last-layer 5.0 \
    --syncbn \
    --prefix module.encoder. \
    --labels-perc 1 \
    --pretrained outputs/jigclu_pretrain/model_best.pth.tar \
    --save-dir outputs/jigclu_semi_1p/ \
    datasets/ImageNet/

Transfer to COCO detection

Please convert the pretrained weight first

python detection/convert.py

Then start training using

python detection/train_net.py --config-file detection/configs/R50-JigClu.yaml --num-gpus 4

VOC detection

python detection/train_net.py --config-file detection/configs/voc-R50-JigClu.yaml --num-gpus 4

Citation

Please consider citing JigsawClustering in your publications if it helps your research.

@inproceedings{chen2021jigclu,
    title={Jigsaw Clustering for Unsupervised Visual Representation Learning},
    author={Pengguang Chen, Shu Liu, and Jiaya Jia},
    booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2021},
}

Comments

Some question about trainning

Hi~Thanks for your excellent work! I have a machine with 2 1080Ti,and I want to train your model on CIFAR10 with resnet18.

I use the parmeters like this ,but it seems don't work.

The program is stuck in this situation.

opened by zbw0329 10
Some details about the training

Hi, I have recently read your paper and find it very interesting. There are still some confusions about the experiments.

The experiments require 4 2080ti for training. Does it mean we must have 4 2080ti on one single machine? What if I have 4 2080ti on different machines? Is there any suggestion for this situation? BTW, how long does it take when you train on ImageNet1k?

Much appreciation for your reply.

Best wishes!

opened by Hanzy1996 3
Some questions about the results of ImageNet100

Thank you for your wonderful work, I want to do some more works based on your code. But I meet some questions about the results. I use the JigsawClustering and the dataset ImageNet100 to train the model. I only changed one line in the model to fit this dataset(I added model.fc = nn.Linear(2048, 100) in line 162 of main_lincls.py). However, despite using 4 GPUs, and did not change the configuration file. I only got an accuracy of 79.24. There is still a certain gap between this and the 80.9 reported in the paper. How can I achieve the accuracy reported in the paper now? Once again, thank you for your excellent work and code. I am looking forward to your reply.

opened by WilyZhao8 1
Results of Faster-RCNN R50-FPN with model pretrained on ImageNet with standard cross-entropy loss

Hi, thanks for your work! In Objection Detection, do you apply ResNet-50 model pretrained on ImageNet with standard cross-entropy loss to Faster-RCNN R50-FPN?

opened by fzfs 1
Training the model on a single GPU

Hi! I'm aware that the question has been asked previously, but could you guide how to modify jigclu to remove the distributeddataparallel depedency?

Thanks!

opened by shuvam-creditmate 2
It seems that the model has not learned anything，What should I do？

Thanks for your excellent work! I change the dataloader to use JigClu in CIFAR-10,and train the model on it by 1000epoch. But the prediction of my model is all the same. It seem that model always cluster into the same cluster

opened by zbw0329 10

Releases(1.0)

1.0(May 11, 2021)

Release pretrained models.
Source code(tar.gz)
Source code(zip)
JigClu_200e.pth(213.72 MB)

Owner

DV Lab

Deep Vision Lab

GitHub Repository

An All-MLP solution for Vision, from Google AI

MLP Mixer - Pytorch An All-MLP solution for Vision, from Google AI, in Pytorch. No convolutions nor attention needed! Yannic Kilcher video Install $ p

784 Jan 06, 2023

Server files for UltimateLabeling

UltimateLabeling server files Server files for UltimateLabeling. git clone https://github.com/alexandre01/UltimateLabeling_server.git cd UltimateLabel

4 Oct 10, 2022

ScaleNet: A Shallow Architecture for Scale Estimation

ScaleNet: A Shallow Architecture for Scale Estimation Repository for the code of ScaleNet paper: "ScaleNet: A Shallow Architecture for Scale Estimatio

34 Nov 09, 2022

Using PyTorch Perform intent classification using three different models to see which one is better for this task

1 Feb 14, 2022

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Deformable Attention Implementation of Deformable Attention from this paper in Pytorch, which appears to be an improvement to what was proposed in DET

128 Dec 24, 2022

SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning

SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning This repository is the official implementation of "SHRIMP: Sparser Random Featur

0 Dec 16, 2021

Imaging, analysis, and simulation software for radio interferometry

ehtim (eht-imaging) Python modules for simulating and manipulating VLBI data and producing images with regularized maximum likelihood methods. This ve

5.2k Dec 28, 2022

Local Multi-Head Channel Self-Attention for FER2013

LHC-Net Local Multi-Head Channel Self-Attention This repository is intended to provide a quick implementation of the LHC-Net and to replicate the resu

12 Jan 04, 2023

Existing Literature about Machine Unlearning

Machine Unlearning Papers 2021 Brophy and Lowd. Machine Unlearning for Random Forests. In ICML 2021. Bourtoule et al. Machine Unlearning. In IEEE Symp

213 Jan 08, 2023

Social Fabric: Tubelet Compositions for Video Relation Detection

Social-Fabric Social Fabric: Tubelet Compositions for Video Relation Detection This repository contains the code and results for the following paper:

7 Aug 09, 2022

Image Captioning using CNN ,LSTM and Attention

Image Captioning using CNN ,LSTM and Attention This is a deeplearning model which tries to summarize an image into a text . Installation Install this

1 Dec 16, 2021

Pytorch implementation of Generative Models as Distributions of Functions 🌿

Generative Models as Distributions of Functions This repo contains code to reproduce all experiments in Generative Models as Distributions of Function

117 Dec 29, 2022

[ICML 2020] DrRepair: Learning to Repair Programs from Error Messages

DrRepair: Learning to Repair Programs from Error Messages This repo provides the source code & data of our paper: Graph-based, Self-Supervised Program

155 Jan 08, 2023

Predicting Tweet Sentiment Maching Learning and streamlit

Predicting-Tweet-Sentiment-Maching-Learning-and-streamlit (I prefere using Visual Studio Code ) Open the folder in VS Code Run the first cell in requi

1 Nov 20, 2021

MVFNet: Multi-View Fusion Network for Efficient Video Recognition (AAAI 2021)

MVFNet: Multi-View Fusion Network for Efficient Video Recognition (AAAI 2021) Overview We release the code of the MVFNet (Multi-View Fusion Network).

2 Jan 29, 2022

PyJokes - Joking around with Python library pyjokes

Hi, it's Muhaimin again 👋 This is something unorthodox but cool. Don't forget t

1 Feb 02, 2022

ML-based medical imaging using Azure

Disclaimer This code is provided for research and development use only. This code is not intended for use in clinical decision-making or for any other

68 Dec 23, 2022

Repository for reproducing `Model-Based Robust Deep Learning`

Model-Based Robust Deep Learning (MBRDL) In this repository, we include the code necessary for reproducing the code used in Model-Based Robust Deep Le

16 Sep 19, 2022

Demystifying How Self-Supervised Features Improve Training from Noisy Labels

Demystifying How Self-Supervised Features Improve Training from Noisy Labels This code is a PyTorch implementation of the paper "[Demystifying How Sel

[email protected]"> 4 Oct 14, 2022

This repo contains the implementation of YOLOv2 in Keras with Tensorflow backend.

Easy training on custom dataset. Various backends (MobileNet and SqueezeNet) supported. A YOLO demo to detect raccoon run entirely in brower is accessible at https://git.io/vF7vI (not on Windows).

1.7k Dec 24, 2022