RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

An AFL implementation with UnTracer (our coverage-guided tracer)

DuBE: Duple-balanced Ensemble Learning from Skewed Data

Wenet STT Python

Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

Official Implementation for HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing

Semi-supervised Transfer Learning for Image Rain Removal. In CVPR 2019.

Code for the paper "Improving Vision-and-Language Navigation with Image-Text Pairs from the Web" (ECCV 2020)

This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data

PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

Code examples and benchmarks from the paper "Understanding Entropy Coding With Asymmetric Numeral Systems (ANS): a Statistician's Perspective"

Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels

Competitive Programming Club, Clinify's Official repository for CP problems hosting by club members.

Safe Local Motion Planning with Self-Supervised Freespace Forecasting, CVPR 2021

Pseudo-rng-app - whos needs science to make a random number when you have pseudoscience?

Plaything for Autistic Children (demo for PaddlePaddle/Wechaty/Mixlab project)

Speeding-Up Back-Propagation in DNN: Approximate Outer Product with Memory

Implement A3C for Mujoco gym envs

House3D: A Rich and Realistic 3D Environment

Official codebase for "B-Pref: Benchmarking Preference-BasedReinforcement Learning" contains scripts to reproduce experiments.