STBP is a way to train SNN with datasets by Backward propagation.

Last update: Dec 09, 2022

Overview

Train SNN with STBP in fp32 and low bit(quantize)

Spiking neural network (SNN), compared with depth neural network (DNN), has faster processing speed, lower energy consumption and more biological interpretability, which is expected to approach Strong AI.

STBP is a way to train SNN with datasets by Backward propagation.Using this Repositories allows you to train SNNS with STBP and quantize SNNS with QAT to deploy to neuromorphological chips like Loihi and Tianjic.

Usage

Download via GitHub:

git clone https://github.com/ZLkanyo009/STBP-train-and-compression.git

example to define SNN_layers like ANN_layers

Convert layer to spatiotemporal layer:

conv = nn.Conv2d(...)
conv_s = tdLayer(conv)

Define LIF activation function just like Relu:

spike = LIFSpike()

In the forward function, replace the activation function of each layer with LIF activation function, and replace the calls such as conv() with conv_ s(), then SNN_layers definition is completed.Finally, we use Frequency Coding to decode SNN's output like out = torch.sum(x, dim=2) / steps

def forward(self, x):
    x = self.conv1_s(x)
    x = self.spike(x)
    x = self.pool1_s(x)
    x = self.spike(x)
    x = x.view(x.shape[0], -1, x.shape[4])
    x = self.fc1_s(x)
    x = self.spike(x)
    out = torch.sum(x, dim=2) / steps
    return out

If BN layer is required:

bn = nn.BatchNorm2d(...)
bn = tdBatchNorm(...)
conv_s = tdLayer(conv, bn)

Training Fp32 Model

# Start training fp32 model with: 
# model_name can be ResNet18, CifarNet, ...
python main.py ResNet18 --dataset CIFAR10

# training with DDP:
python -m torch.distributed.launch main.py ResNet18 --local_rank 0 --dataset CIFAR10 --p DDP

# You can manually config the training with: 
python main.py ResNet18 --resume --lr 0.01

Training Quantize Model

# Start training quantize model with: 
# model_name can be ResNet18, CifarNet, ...
python main.py ResNet18 --dataset CIFAR10 -q

# training with DDP:
python -m torch.distributed.launch main.py ResNet18 --local_rank 0 --dataset CIFAR10 -q --p DDP

# You can manually config the training with: 
python main.py ResNet18 -q --resume --bit 4 --lr 0.01

Accuracy

Model	Acc.(fp32)	Acc.(8 bit quantize)
MNISTNet	97.96%	97.57%
ResNet18	84.40%	84.23%

STBP is a way to train SNN with datasets by Backward propagation.

Related tags

Overview

Train SNN with STBP in fp32 and low bit(quantize)

Usage

Download via GitHub:

example to define SNN_layers like ANN_layers

Training Fp32 Model

Training Quantize Model

Accuracy

About STBP

Owner

Ling Zhang

Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval (NeurIPS'21)

Head and Neck Tumour Segmentation and Prediction of Patient Survival Project

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

Pipeline code for Sequential-GAM(Genome Architecture Mapping).

How to train a CNN to 99% accuracy on MNIST in less than a second on a laptop

VOneNet: CNNs with a Primary Visual Cortex Front-End

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Res2Net for Instance segmentation and Object detection using MaskRCNN

SimulLR - PyTorch Implementation of SimulLR

Markov Attention Models

Repository for "Improving evidential deep learning via multi-task learning," published in AAAI2022

PanopticBEV - Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images

Code for "Learning the Best Pooling Strategy for Visual Semantic Embedding", CVPR 2021

Code for "LoFTR: Detector-Free Local Feature Matching with Transformers", CVPR 2021

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Repositorio oficial del curso IIC2233 Programación Avanzada 🚀✨

Collect some papers about transformer with vision. Awesome Transformer with Computer Vision (CV)

An unopinionated replacement for PyTorch's Dataset and ImageFolder, that handles Tar archives

This package implements the algorithms introduced in Smucler, Sapienza, and Rotnitzky (2020) to compute optimal adjustment sets in causal graphical models.

[AAAI2021] The source code for our paper 《Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion》.