Skip to content

NeurIPS 2021, Official codes for "Efficient Training of Visual Transformers with Small Datasets".

License

Notifications You must be signed in to change notification settings

yhlleo/VTs-Drloc

Repository files navigation

Efficient Training of Visual Transformers with Small Datasets

Maintenance Contributing

To appear in NerIPS 2021.

[paper][Poster & Video][arXiv][code] [reviews]
Yahui Liu1,3, Enver Sangineto1, Wei Bi2, Nicu Sebe1, Bruno Lepri3, Marco De Nadai3
1University of Trento, Italy, 2Tencent AI Lab, China, 3Bruno Kessler Foundation, Italy.

Data preparation

Dataset Download Link
ImageNet train,val
CIFAR-10 all
CIFAR-100 all
SVHN train,test, extra
Oxford-Flower102 images, labels, splits
Clipart images, train_list, test_list
Infograph images, train_list, test_list
Painting images, train_list, test_list
Quickdraw images, train_list, test_list
Real images, train_list, test_list
Sketch images, train_list, test_list
  • Download the datasets and pre-processe some of them (i.e., imagenet, domainnet) by using codes in the scripts folder.
  • The datasets are prepared with the following stucture (except CIFAR-10/100 and SVHN):
dataset_name
  |__train
  |    |__category1
  |    |    |__xxx.jpg
  |    |    |__...
  |    |__category2
  |    |    |__xxx.jpg
  |    |    |__...
  |    |__...
  |__val
       |__category1
       |    |__xxx.jpg
       |    |__...
       |__category2
       |    |__xxx.jpg
       |    |__...
       |__...

Training

After prepare the datasets, we can simply start the training with 8 NVIDIA V100 GPUs:

sh train.sh

Evaluation

We can also load the pre-trained model and test the performance:

sh eval.sh

Pretrained models

For fast evaluation, we present the results of Swin-T trained with 100 epochs on various datasets as an example (Note that we save the model every 5 epochs during the training, so the attached best models may be slight different from the reported performances).

Datasets Baseline Ours
CIFAR-10 59.47 83.89
CIFAR-100 53.28 66.23
SVHN 71.60 94.23
Flowers102 34.51 39.37
Clipart 38.05 47.47
Infograph 8.20 10.16
Painting 35.92 41.86
Quickdraw 24.08 69.41
Real 73.47 75.59
Sketch 11.97 38.55

We provide a demo to download the pretrained models from Google Drive directly:

python3 ./scripts/collect_models.py

Related Work:

Acknowledgments

This code is highly based on the Swin-Transformer. Thanks to the contributors of this project.

Citation

@InProceedings{liu2021efficient,
    author    = {Liu, Yahui and Sangineto, Enver and Bi, Wei and Sebe, Nicu and Lepri, Bruno and De Nadai, Marco},
    title     = {Efficient Training of Visual Transformers with Small Datasets},
    booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
    year      = {2021}
}

If you have any questions, please contact me without hesitation (yahui.cvrs AT gmail.com).

About

NeurIPS 2021, Official codes for "Efficient Training of Visual Transformers with Small Datasets".

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published