RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

[CVPR 2022] TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing

Iranian Cars Detection using Yolov5s, PyTorch

Nsdf: A mesh SDF with just some code we can directly paste into our raymarcher

An open source implementation of CLIP.

The codes and related files to reproduce the results for Image Similarity Challenge Track 2.

Few-shot Learning of GPT-3

Concept drift monitoring for HA model servers.

AI4Good project for detecting waste in the environment

Code for Environment Dynamics Decomposition (ED2).

Differentiable Annealed Importance Sampling (DAIS)

Quadruped-command-tracking-controller - Quadruped command tracking controller (flat terrain)

This repository is a basic Machine Learning train & validation Template (Using PyTorch)

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

The code for MM2021 paper "Multi-Level Counterfactual Contrast for Visual Commonsense Reasoning"

Opinionated code formatter, just like Python's black code formatter but for Beancount

System Combination for Grammatical Error Correction Based on Integer Programming

Implementation of "The Power of Scale for Parameter-Efficient Prompt Tuning"

Learning Skeletal Articulations with Neural Blend Shapes

Code and data form the paper BERT Got a Date: Introducing Transformers to Temporal Tagging

Visual dialog agents with pre-trained vision-and-language encoders.