Bottleneck Transformers for Visual Recognition

Last update: Jan 03, 2023

Overview

Bottleneck Transformers for Visual Recognition

Experiments

Model	Params (M)	Acc (%)
ResNet50 baseline (ref)	23.5M	93.62
BoTNet-50	18.8M	95.11%
BoTNet-S1-50	18.8M	95.67%
BoTNet-S1-59	27.5M	95.98%
BoTNet-S1-77	44.9M	wip

Summary

Usage (example)

Model

from model import Model

model = ResNet50(num_classes=1000, resolution=(224, 224))
x = torch.randn([2, 3, 224, 224])
print(model(x).size())

Module

from model import MHSA

resolution = 14
mhsa = MHSA(planes, width=resolution, height=resolution)

Reference

Paper link
Author: Aravind Srinivas, Tsung-Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, Ashish Vaswani
Organization: UC Berkeley, Google Research

Owner

Myeongjun Kim

Computer Vision Research using Deep Learning

GitHub Repository

Head2Toe: Utilizing Intermediate Representations for Better OOD Generalization

Head2Toe: Utilizing Intermediate Representations for Better OOD Generalization Code for reproducing our results in the Head2Toe paper. Paper: arxiv.or

62 Dec 12, 2022

Libraries, tools and tasks created and used at DeepMind Robotics.

dm_robotics: Libraries, tools, and tasks created and used for Robotics research at DeepMind. Package overview Package Summary Transformations Rigid bo

273 Jan 06, 2023

Code for project: "Learning to Minimize Remainder in Supervised Learning".

Learning to Minimize Remainder in Supervised Learning Code for project: "Learning to Minimize Remainder in Supervised Learning". Requirements and Envi

0 Jul 18, 2021

Instance-based label smoothing for improving deep neural networks generalization and calibration

Instance-based Label Smoothing for Neural Networks Pytorch Implementation of the algorithm. This repository includes a new proposed method for instanc

1 Aug 13, 2022

Code for the Weighted, Accelerated and Restarted Primal-dual algorithm. This algorithm achieves stable linear convergence for reconstruction from undersampled noisy measurements under an approximate sharpness condition. See the paper for details.

WARPd Code for the Weighted, Accelerated and Restarted Primal-dual algorithm. This algorithm achieves stable linear convergence for reconstruction fro

1 Apr 08, 2022

Neuron Merging: Compensating for Pruned Neurons (NeurIPS 2020)

Neuron Merging: Compensating for Pruned Neurons Pytorch implementation of Neuron Merging: Compensating for Pruned Neurons, accepted at 34th Conference

33 Dec 30, 2022

[SIGGRAPH 2022 Journal Track] AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars Fangzhou Hong1* Mingyuan Zhang1* Liang Pan1 Zhongang Cai1,2,3 Lei Yang2

749 Jan 04, 2023

Signals-backend - A suite of card games written in Python

Card game A suite of card games written in the Python language. Features coming

1 Feb 15, 2022

Pointer-generator - Code for the ACL 2017 paper Get To The Point: Summarization with Pointer-Generator Networks

Note: this code is no longer actively maintained. However, feel free to use the Issues section to discuss the code with other users. Some users have u

2.1k Jan 04, 2023

Cave Generation using metaballs in Blender. Originally created by sdfgeoff, Edited by Myself (Archie Jaskowicz).

Blender-Cave-Generation Cave Generation using metaballs in Blender. Originally created by sdfgeoff, Edited by Myself (Archie Jaskowicz). Installation

2 Dec 28, 2022

Pytorch implementation of "Training a 85.4% Top-1 Accuracy Vision Transformer with 56M Parameters on ImageNet"

Token Labeling: Training an 85.4% Top-1 Accuracy Vision Transformer with 56M Parameters on ImageNet (arxiv) This is a Pytorch implementation of our te

383 Dec 27, 2022

[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Using Unreliable Pseudo Labels Official PyTorch implementation of Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels, CVPR 2022. Ple

268 Dec 24, 2022

Solutions of Reinforcement Learning 2nd Edition

Solutions of Reinforcement Learning, An Introduction

1.4k Dec 30, 2022

Code to train models from "Paraphrastic Representations at Scale".

Paraphrastic Representations at Scale Code to train models from "Paraphrastic Representations at Scale". The code is written in Python 3.7 and require

71 Dec 19, 2022

Unet network with mean teacher for altrasound image segmentation

5 Nov 21, 2022

A library for performing coverage guided fuzzing of neural networks

TensorFuzz: Coverage Guided Fuzzing for Neural Networks This repository contains a library for performing coverage guided fuzzing of neural networks,

195 Dec 28, 2022

Code for the paper "Reinforced Active Learning for Image Segmentation"

Reinforced Active Learning for Image Segmentation (RALIS) Code for the paper Reinforced Active Learning for Image Segmentation Dependencies python 3.6

79 Dec 19, 2022

The Submission for SIMMC 2.0 Challenge 2021

The Submission for SIMMC 2.0 Challenge 2021 challenge website Requirements python 3.8.8 pytorch 1.8.1 transformers 4.8.2 apex for multi-gpu nltk Prepr

5 Jul 26, 2022

Implementation of paper "DCS-Net: Deep Complex Subtractive Neural Network for Monaural Speech Enhancement"

DCS-Net This is the implementation of "DCS-Net: Deep Complex Subtractive Neural Network for Monaural Speech Enhancement" Steps to run the model Edit V

10 Apr 04, 2022

This is the implementation of our work Deep Extreme Cut (DEXTR), for object segmentation from extreme points.

828 Jan 05, 2023

Bottleneck Transformers for Visual Recognition

Related tags

Overview

Bottleneck Transformers for Visual Recognition

Experiments

Summary

Usage (example)

Reference

Owner

Myeongjun Kim

Head2Toe: Utilizing Intermediate Representations for Better OOD Generalization

Libraries, tools and tasks created and used at DeepMind Robotics.

Code for project: "Learning to Minimize Remainder in Supervised Learning".

Instance-based label smoothing for improving deep neural networks generalization and calibration

Code for the Weighted, Accelerated and Restarted Primal-dual algorithm. This algorithm achieves stable linear convergence for reconstruction from undersampled noisy measurements under an approximate sharpness condition. See the paper for details.

Neuron Merging: Compensating for Pruned Neurons (NeurIPS 2020)

[SIGGRAPH 2022 Journal Track] AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Signals-backend - A suite of card games written in Python

Pointer-generator - Code for the ACL 2017 paper Get To The Point: Summarization with Pointer-Generator Networks

Cave Generation using metaballs in Blender. Originally created by sdfgeoff, Edited by Myself (Archie Jaskowicz).

Pytorch implementation of "Training a 85.4% Top-1 Accuracy Vision Transformer with 56M Parameters on ImageNet"

[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Solutions of Reinforcement Learning 2nd Edition

Code to train models from "Paraphrastic Representations at Scale".

Unet network with mean teacher for altrasound image segmentation

A library for performing coverage guided fuzzing of neural networks

Code for the paper "Reinforced Active Learning for Image Segmentation"

The Submission for SIMMC 2.0 Challenge 2021

Implementation of paper "DCS-Net: Deep Complex Subtractive Neural Network for Monaural Speech Enhancement"

This is the implementation of our work Deep Extreme Cut (DEXTR), for object segmentation from extreme points.