An addernet CUDA version

Last update: Jun 20, 2022

Related tags

Overview

Training addernet accelerated by CUDA

Usage

cd adder_cuda
python setup.py install
cd ..
python main.py

Environment

pytorch 1.10.0 CUDA 11.3

benchmark

version	training_time_per_batch/s
raw	1.61
torch.cdist	1.49
cuda_unoptimized	0.4508
this work	0.3158

The CUDA version of AdderNet has achieved a 5× speed increase over the original version. There seems to be some bugs in the Cuda_unoptimized version, causing the model to fail to converge. Its speed is still listed here for comparison. The experiment was run on RTX 2080Ti platform, and ResNet-20 based on CIFAR-10 was trained.

Time(%)	Time	Calls	Avg	Min	Max	Name
48.57	30.4752s	3920	7.7743ms	162.70us	12.271ms	CONV_BACKWARD
34.85	21.8686s	19680	1.1112ms	5.3770us	11.827ms	_ZN2at6native27unrolled_elementwise_kernel...
7.46	4.67901s	5920	790.37us	26.529us	1.5841ms	CONV
2.24	1.40372s	3920	358.09us	31.298us	845.80us	col2im_kernel
2.10	1.31882s	36862	35.777us	1.4720us	276.24us	vectorized_elementwise_kernel
1.43	900.03ms	5920	152.03us	7.9040us	372.40us	im2col_kernel

Here is the time distribution of training an epoch. If you are interested, you can continue to optimize the CUDA kernel.

An addernet CUDA version

Related tags

Overview

Training addernet accelerated by CUDA

Usage

Environment

benchmark

Owner

LingXY

[CVPR 2022 Oral] MixFormer: End-to-End Tracking with Iterative Mixed Attention

Yolo object detection - Yolo object detection with python

Updated for TTS(CE) = Also Known as TTN V3. The code requires the first server to be 'ttn' protocol.

3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks

Stochastic Scene-Aware Motion Prediction

This is the official PyTorch implementation of our paper: "Artistic Style Transfer with Internal-external Learning and Contrastive Learning".

A PyTorch based deep learning library for drug pair scoring.

Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].

PyTorch implementation of Higher Order Recurrent Space-Time Transformer

Code & Data for Enhancing Photorealism Enhancement

Implements an infinite sum of poisson-weighted convolutions

Official codebase for Decision Transformer: Reinforcement Learning via Sequence Modeling.

Perfect implement. Model shared. x0.5 (Top1:60.646) and 1.0x (Top1:69.402).

Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics

The Pytorch code of "Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification", CVPR 2022 (Oral).

Model Quantization Benchmark

PyTorch GPU implementation of the ES-RNN model for time series forecasting

CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation

UNAVOIDS: Unsupervised and Nonparametric Approach for Visualizing Outliers and Invariant Detection Scoring

Code for the paper "SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness" (NeurIPS 2021)