Consistency Regularization for Adversarial Robustness

Official PyTorch implementation of Consistency Regularization for Adversarial Robustness by Jihoon Tack, Sihyun Yu, Jongheon Jeong, Minseon Kim, Sung Ju Hwang, and Jinwoo Shin.

1. Dependencies

conda create -n con-adv python=3
conda activate con-adv

conda install pytorch torchvision cudatoolkit=11.0 -c pytorch 

pip install git+https://github.com/fra31/auto-attack
pip install advertorch tensorboardX

2. Training

2.1. Training option and description

The option for the training method is as follows:

• <DATASET>: {cifar10,cifar100,tinyimagenet}
• <AUGMENT>: {base,ccg}
• <ADV_TRAIN OPTION>: {adv_train,adv_trades,adv_mart}

Current code are assuming l_infinity constraint adversarial training and PreAct-ResNet-18 as a base model.
To change the option, simply modify the following configurations:

• WideResNet-34-10: --model wrn3410
• l_2 constraint: --distance L2

2.2. Training code

Standard cross-entropy training

% Standard cross-entropy
python train.py --mode ce --augment base --dataset <DATASET>

Adversarial training

% Adversarial training
python train.py --mode <ADV_TRAIN OPTION> --augment <AUGMENT> --dataset <DATASET>

% Example: Standard AT under CIFAR-10
python train.py --mode adv_train --augment base --dataset cifar10

Consistency regularization

% Consistency regularization
python train.py --consistency --mode <ADV_TRAIN OPTION> --augment <AUGMENT> --dataset <DATASET>

% Example: Consistency regularization based on standard AT under CIFAR-10
python train.py --consistency --mode adv_train --augment ccg --dataset cifar10 

3. Evaluation

3.1. Evaluation option and description

The description for treat model is as follows:

• <DISTANCE>: {Linf,L2,L1}, the norm constraint type
• <EPSILON>: the epsilon ball size
• <ALPHA>: the step size of PGD optimization
• <NUM_ITER>: iteration number of PGD optimization

3.2. Evaluation code

Evaluate clean accuracy

python eval.py --mode test_clean_acc --dataset <DATASET> --load_path <MODEL_PATH>

Evaluate clean & robust accuracy against PGD

python eval.py --mode test_adv_acc --distance <DISTANCE> --epsilon <EPSILON> --alpha <ALPHA> --n_iters <NUM_ITER> --dataset <DATASET> --load_path <MODEL_PATH>

Evaluate clean & robust accuracy against AutoAttack

python eval.py --mode test_auto_attack --epsilon <EPSILON> --distance <DISTANCE> --dataset <DATASET> --load_path <MODEL_PATH>

Evaluate mean corruption error (mCE)

python eval.py --mode test_mce --dataset <DATASET> --load_path <MODEL_PATH>

4. Results

White box attack

Clean accuracy and robust accuracy (%) against white-box attacks on PreAct-ResNet-18 trained on CIFAR-10.
We use l_infinity threat model with epsilon = 8/255.

Unseen adversaries

Robust accuracy (%) of PreAct-ResNet-18 trained with of l_infinity epsilon = 8/255 constraint against unseen attacks.
For unseen attacks, we use PGD-100 under different sized l_infinity epsilon balls, and other types of norm balls.

Mean corruption error

Mean corruption error (mCE) (%) of PreAct-ResNet-18 trained on CIFAR-10, and tested with CIFAR-10-C dataset

Reference

• TRADES
• MART