RefineMask (CVPR 2021)

Related tags

Deep Learninginstance-segmentation
Overview

RefineMask: Towards High-Quality Instance Segmentation
with Fine-Grained Features (CVPR 2021)

This repo is the official implementation of RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features.

Framework

image

Main Results

Results on COCO

Method Backbone Schedule AP AP* Checkpoint
Mask R-CNN R50-FPN 1x 34.7 36.8
RefineMask R50-FPN 1x 37.3 40.6 download
Mask R-CNN R50-FPN 2x 35.4 37.7
RefineMask R50-FPN 2x 37.8 41.2 download
Mask R-CNN R101-FPN 1x 36.1 38.4
RefineMask R101-FPN 1x 38.6 41.8 download
Mask R-CNN R101-FPN 2x 36.6 39.3
RefineMask R101-FPN 2x 39.0 42.4 download

Note: No data augmentations except standard horizontal flipping were used.

Results on LVIS

Method Backbone Schedule AP APr APc APf Checkpoint
Mask R-CNN R50-FPN 1x 22.1 10.1 21.7 30.0
RefineMask R50-FPN 1x 25.7 13.8 24.9 31.8 download
Mask R-CNN R101-FPN 1x 23.7 12.3 23.2 29.1
RefineMask R101-FPN 1x 27.1 15.6 26.2 33.1 download

Results on Cityscapes

Method Backbone Schedule AP APS APM APL Checkpoint
Mask R-CNN R50-FPN 1x 33.8 12.0 31.5 51.8
RefineMask R50-FPN 1x 37.6 14.0 35.4 57.9 download

Efficiency of RefineMask

Method AP AP* FPS
Mask R-CNN 34.7 36.8 15.7
PointRend 35.6 38.7 11.4
HTC 37.4 40.7 4.4
RefineMask 37.3 40.9 11.4

Usage

Requirements

  • Python 3.6+
  • Pytorch 1.5.0
  • mmcv-full 1.0.5

Datasets

data
  ├── coco
  |   ├── annotations
  │   │   │   ├── instances_train2017.json
  │   │   │   ├── instances_val2017.json
  │   │   │   ├── lvis_v0.5_val_cocofied.json
  │   ├── train2017
  │   │   ├── 000000004134.png
  │   │   ├── 000000031817.png
  │   │   ├── ......
  │   ├── val2017
  │   ├── test2017
  ├── lvis
  |   ├── annotations
  │   │   │   ├── lvis_v1_train.json
  │   │   │   ├── lvis_v1_val.json
  │   ├── train2017
  │   │   ├── 000000004134.png
  │   │   ├── 000000031817.png
  │   │   ├── ......
  │   ├── val2017
  │   ├── test2017
  ├── cityscapes
  |   ├── annotations
  │   │   │   ├── instancesonly_filtered_gtFine_train.json
  │   │   │   ├── instancesonly_filtered_gtFine_val.json
  │   ├── leftImg8bit
  │   |   ├── train
  │   │   ├── val
  │   │   ├── test

Note: We used the lvis-v1.0 dataset which consists of 1203 categories.

Training

./scripts/dist_train.sh ./configs/refinemask/coco/r50-refinemask-1x.py 8

Note: The codes only support batch size 1 per GPU, and we trained all models with a total batch size 16x1. If you train models with a total batch size 8x1, the performance may drop. We will support batch size 2 or more per GPU later. You can use ./scripts/slurm_train.sh for training with multi-nodes.

Inference

./scripts/dist_test.sh ./configs/refinemask/coco/r50-refinemask-1x.py xxxx.pth 8

Citation

@article{zhang2021refinemask,
  title={RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features},
  author={Gang, Zhang and Xin, Lu and Jingru, Tan and Jianmin, Li and Zhaoxiang, Zhang and Quanquan, Li and Xiaolin, Hu},
  journal={arXiv preprint arXiv:2104.08569},
  year={2021}
}
Owner
Gang Zhang
Ph.D. student in Tsinghua University [email protected]
Gang Zhang
GitHub Repository
A practical ML pipeline for data labeling with experiment tracking using DVC.

Auto Label Pipeline A practical ML pipeline for data labeling with experiment tracking using DVC Goals: Demonstrate reproducible ML Use DVC to build a

Todd Cook 4 Mar 08, 2022
Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust.

Subspace Adversarial Training Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust. However,

15 Sep 02, 2022
Learning a mapping from images to psychological similarity spaces with neural networks.

LearningPsychologicalSpaces v0.1: v1.1: v1.2: v1.3: v1.4: v1.5: The code in this repository explores learning a mapping from images to psychological s

Lucas Bechberger 8 Dec 12, 2022
Simulation code and tutorial for BBHnet training data

Simulation Dataset for BBHnet NOTE: OLD README, UPDATE IN PROGRESS We generate simulation dataset to train BBHnet, our deep learning framework for det

0 May 31, 2022
Steerable discovery of neural audio effects

Steerable discovery of neural audio effects Christian J. Steinmetz and Joshua D. Reiss Abstract Applications of deep learning for audio effects often

Christian J. Steinmetz 182 Dec 29, 2022
This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

A three-stage detection and recognition pipeline of complex meters in wild This is the first released system towards detection and recognition of comp

Yan Shu 19 Nov 28, 2022
The "breathing k-means" algorithm with datasets and example notebooks

The Breathing K-Means Algorithm (with examples) The Breathing K-Means is an approximation algorithm for the k-means problem that (on average) is bette

Bernd Fritzke 75 Nov 17, 2022
Benchmarks for the Optimal Power Flow Problem

Power Grid Lib - Optimal Power Flow This benchmark library is curated and maintained by the IEEE PES Task Force on Benchmarks for Validation of Emergi

A Library of IEEE PES Power Grid Benchmarks 207 Dec 08, 2022
Pytorch implementation of the paper "COAD: Contrastive Pre-training with Adversarial Fine-tuning for Zero-shot Expert Linking."

Expert-Linking Pytorch implementation of the paper "COAD: Contrastive Pre-training with Adversarial Fine-tuning for Zero-shot Expert Linking." This is

BoChen 12 Jan 01, 2023
PyTorch code for: Learning to Generate Grounded Visual Captions without Localization Supervision

Learning to Generate Grounded Visual Captions without Localization Supervision This is the PyTorch implementation of our paper: Learning to Generate G

Chih-Yao Ma 41 Nov 17, 2022
BEGAN in PyTorch

BEGAN in PyTorch This project is still in progress. If you are looking for the working code, use BEGAN-tensorflow. Requirements Python 2.7 Pillow tqdm

Taehoon Kim 260 Dec 07, 2022
The 2nd place solution of 2021 google landmark retrieval on kaggle.

Google_Landmark_Retrieval_2021_2nd_Place_Solution The 2nd place solution of 2021 google landmark retrieval on kaggle. Environment We use cuda 11.1/pyt

229 Dec 13, 2022
The code repository for EMNLP 2021 paper "Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization".

Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization [Paper] accepted at the EMNLP 2021: Vision Guided Genera

CAiRE 42 Jan 07, 2023
StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion Yinghao Aaron Li, Ali Zare, Nima Mesgarani We pres

Aaron (Yinghao) Li 282 Jan 01, 2023
This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022).

MoEBERT This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022). Installation Create an

Simiao Zuo 34 Dec 24, 2022
Catbird is an open source paraphrase generation toolkit based on PyTorch.

Catbird is an open source paraphrase generation toolkit based on PyTorch. Quick Start Requirements and Installation The project is based on PyTorch 1.

Afonso Salgado de Sousa 5 Dec 15, 2022
Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.

HiddenLayer A lightweight library for neural network graphs and training metrics for PyTorch, Tensorflow, and Keras. HiddenLayer is simple, easy to ex

Waleed 1.7k Dec 31, 2022
The VarCNN is an Convolution Neural Network based approach to automate Video Assistant Referee in football.

VarCnn: The Deep Learning Powered VAR

Aamir 1 Jan 09, 2022
Official release of MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis of Pancreatic Cancer axriv: http://arxiv.org/abs/2112.13513

MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis This is the official page of the MSHT with its experimental script and records. We de

Tianyi Zhang 53 Dec 27, 2022
I-BERT: Integer-only BERT Quantization

I-BERT: Integer-only BERT Quantization HuggingFace Implementation I-BERT is also available in the master branch of HuggingFace! Visit the following li

Sehoon Kim 139 Dec 27, 2022