Hierarchical-Bayesian-Defense - Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference (Openreview)

Last update: Dec 02, 2022

Overview

Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference [paper]

Baseline of this code is the official repository for this paper. We just replace the BNN regularizer from ELBO with enhanced Bayesian regularizer based on hierarchical-ELBO.

Citation

If you find this work helpful, please cite it as:

@misc{
lee2021towards,
title={Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference},
author={Byung-Kwan Lee and Youngjoon Yu and Yong Man Ro},
year={2021},
url={https://openreview.net/forum?id=Cue2ZEBf12}
}

Hierarchical-Bayeisan-Defense

Dataset

CIFAR10
STL10
CIFAR100
Tiny-ImageNet

Network

VGG16 (for CIFAR-10/CIFAR-100/Tiny-ImageNet)
Aaron (for STL10)
WideResNet (for CIFAR-10/100)

Attack (by torchattack)

PGD attack
EOT-PGD attack

Defense methods

adv: Adversarial training
adv_vi: Adversarial training with Bayesian neural network
adv_hvi: Adversarial training with Enhanced Bayesian neural network based on hierarchical-ELBO

How to Train

1. Adversarial training

Run train_adv.sh

lr=0.01
steps=10
max_norm=0.03
data=tiny # or `cifar10`, `stl10`, `cifar100`
root=./datasets
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
model_out=./checkpoint/${data}_${model}_${max_norm}_adv
echo "Loading: " ${model_out}
CUDA_VISIBLE_DEVICES=0 python ./main_adv.py \
                        --lr ${lr} \
                        --step ${steps} \
                        --max_norm ${max_norm} \
                        --data ${data} \
                        --model ${model} \
                        --root ${root} \
                        --model_out ${model_out}.pth \

2. Adversarial training with BNN

Run train_adv_vi.sh

lr=0.01
steps=10
max_norm=0.03
sigma_0=0.1
init_s=0.1
data=tiny # or `cifar10`, `stl10`, `cifar100`
root=./datasets
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
model_out=./checkpoint/${data}_${model}_${max_norm}_adv_vi
echo "Loading: " ${model_out}
CUDA_VISIBLE_DEVICES=0 python3 ./main_adv_vi.py \
                        --lr ${lr} \
                        --step ${steps} \
                        --max_norm ${max_norm} \
                        --sigma_0 ${sigma_0} \
                        --init_s ${init_s} \
                        --data ${data} \
                        --model ${model} \
                        --root ${root} \
                        --model_out ${model_out}.pth \

3. Adversarial training with enhanced Bayesian regularizer based on hierarchical-ELBO

Run train_adv_hvi.sh

lr=0.01
steps=10
max_norm=0.03
sigma_0=0.1
init_s=0.1
data=tiny # or `cifar10`, `stl10`, `cifar100`
root=./datasets
model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
model_out=./checkpoint/${data}_${model}_${max_norm}_adv_hvi
echo "Loading: " ${model_out}
CUDA_VISIBLE_DEVICES=0 python3 ./main_adv_hvi.py \
                        --lr ${lr} \
                        --step ${steps} \
                        --max_norm ${max_norm} \
                        --sigma_0 ${sigma_0} \
                        --init_s ${init_s} \
                        --data ${data} \
                        --model ${model} \
                        --root ${root} \
                        --model_out ${model_out}.pth \

How to Test

Testing adversarial robustness

Run acc_under_attack.sh

model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
defense=adv_hvi # or `adv_vi`, `adv`
data=tiny-imagenet # or `cifar10`, `stl10`, `cifar100`
root=./datasets
n_ensemble=50
step=10
max_norm=0.03
echo "Loading" ./checkpoint/${data}_${model}_${max_norm}_${defense}.pth

CUDA_VISIBLE_DEVICES=0 python3 acc_under_attack.py \
    --model $model \
    --defense $defense \
    --data $data \
    --root $root \
    --n_ensemble $n_ensemble \
    --step $step \
    --max_norm $max_norm

How to check the learning parameters and KL divergence

Run check_parameters.sh

model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
defense=adv_hvi # or `adv_vi`
data=tiny-imagenet # or `cifar10`, `stl10`, `cifar100`
max_norm=0.03
echo "Loading" ./checkpoint/${data}_${model}_${max_norm}_${defense}.pth

CUDA_VISIBLE_DEVICES=0 python3 check_parameters.py \
    --model $model \
    --defense $defense \
    --data $data \
    --max_norm $max_norm \

How to check uncertainty by predictive entropy

Run uncertainty.sh

model=vgg # vgg for `cifar10` `stl10` `cifar100`, aaron for `stl10`, wide for `cifar10` or `cifar100`
defense=adv_hvi # or `adv_vi`
data=tiny-imagenet # or `cifar10`, `stl10`, `cifar100`
root=./datasets
n_ensemble=50
step=10
max_norm=0.03
echo "Loading" ./checkpoint/${data}_${model}_${max_norm}_${defense}.pth

CUDA_VISIBLE_DEVICES=0 python3 uncertainty.py \
    --model $model \
    --defense $defense \
    --data $data \
    --root $root \
    --n_ensemble $n_ensemble \
    --step $step \
    --max_norm $max_norm

Hierarchical-Bayesian-Defense - Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference (Openreview)

Related tags

Overview

Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference [paper]

Citation

Hierarchical-Bayeisan-Defense

Dataset

Network

Attack (by torchattack)

Defense methods

How to Train

1. Adversarial training

2. Adversarial training with BNN

3. Adversarial training with enhanced Bayesian regularizer based on hierarchical-ELBO

How to Test

Testing adversarial robustness

How to check the learning parameters and KL divergence

How to check uncertainty by predictive entropy

Owner

LBK

An end-to-end machine learning web app to predict rugby scores (Pandas, SQLite, Keras, Flask, Docker)

Python Multi-Agent Reinforcement Learning framework

Unofficial Alias-Free GAN implementation. Based on rosinality's version with expanded training and inference options.

a grammar based feedback fuzzer

A Repository of Community-Driven Natural Instructions

Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

Source code for paper: Knowledge Inheritance for Pre-trained Language Models

My coursework for Machine Learning (2021 Spring) at National Taiwan University (NTU)

Awesome-google-colab - Google Colaboratory Notebooks and Repositories

GAN-STEM-Conv2MultiSlice - Exploring Generative Adversarial Networks for Image-to-Image Translation in STEM Simulation

Implemented fully documented Particle Swarm Optimization algorithm (basic model with few advanced features) using Python programming language

Malware Analysis Neural Network project.

AI Toolkit for Healthcare Imaging

Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)

An open-source online reverse dictionary.

Graph Convolutional Networks for Temporal Action Localization (ICCV2019)

Cancer metastasis detection with neural conditional random field (NCRF)

An LSTM for time-series classification

6D Grasping Policy for Point Clouds

Implementations of orthogonal and semi-orthogonal convolutions in the Fourier domain with applications to adversarial robustness