Few-Shot Object Detection via Association and DIscrimination

Related tags

Deep LearningFADI
Overview

Few-Shot Object Detection via Association and DIscrimination

Code release of our NeurIPS 2021 paper: Few-Shot Object Detection via Association and DIscrimination.

FSCE Figure

Bibtex

@inproceedings{cao2021few,
  title={Few-Shot Object Detection via Association and DIscrimination},
  author={Cao, Yuhang and Wang, Jiaqi and Jin, Ying and Wu, Tong and Chen, Kai and Liu, Ziwei and Lin, Dahua},
  booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
  year={2021}
}

Arxiv: https://arxiv.org/abs/2111.11656

Install dependencies

  • Create a new environment: conda create -n fadi python=3.8 -y
  • Active the newly created environment: conda activate fadi
  • Install PyTorch and torchvision: conda install pytorch=1.7 torchvision cudatoolkit=10.2 -c pytorch -y
  • Install MMDetection: pip install mmdet==2.11.0
  • Install MMCV: pip install mmcv==1.2.5
  • Install MMCV-Full: pip install mmcv-full==1.2.5 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.7.0/index.html

Note:

  • Only tested on MMDet==2.11.0, MMCV==1.2.5, it may not be consistent with other versions.
  • The above instructions use CUDA 10.2, make sure you install the correct PyTorch, Torchvision and MMCV-Full that are consistent with your CUDA version.

Prepare dataset

We follow exact the same split with TFA, please download the dataset and split files as follows:

Create a directory data in the root directory, and the expected structure for data directory:

data/
    VOCdevkit
    few_shot_voc_split

Training & Testing

Base Training

FADI share the same base training stage with TFA, we directly convert the corresponding checkpoints from TFA in Detectron2 format to MMDetection format, please download the base training checkpoints following the table.

Name Split
AP50
download
Base Model 1 80.8 model  | surgery
Base Model 2 81.9 model  | surgery
Base Model 3 82.0 model  | surgery

Create a directory models in the root directory, and the expected structure for models directory:

models/
    voc_split1_base.pth
    voc_split1_base_surgery.pth
    voc_split2_base.pth
    voc_split2_base_surgery.pth
    voc_split3_base.pth
    voc_split3_base_surgery.pth

Few-Shot Fine-tuning

FADI divides the few-shot fine-tuning stage into two steps, ie, association and discrimination,

Suppose we want to train a model for Pascal VOC split1, shot1 with 8 GPUs

1. Step 1: Association.

Getting the assigning scheme of the split:

python tools/associate.py 1

Aligning the feature distribution of the associated base and novel classes:

./tools/dist_train.sh configs/voc_split1/fadi_split1_shot1_association.py 8

2. Step 2: Discrimination

Building a discriminate feature space for novel classes with disentangling and set-specialized margin loss:

./tools/dist_train.sh configs/voc_split1/fadi_split1_shot1_discrimination.py 8

Holistically Training:

We also provide you a script tools/fadi_finetune.sh to holistically train a model for a specific split/shot by running:

./tools/fadi_finetune.sh 1 1

Evaluation

To evaluate the trained models, run

./tools/dist_test.sh configs/voc_split1/fadi_split1_shot1_discrimination.py [checkpoint] 8 --eval mAP --out res.pkl

Model Zoo

Pascal VOC split 1

Shot
nAP50
download
1 50.6 association  | discrimination
2 54.8 association  | discrimination
3 54.1 association  | discrimination
5 59.4 association  | discrimination
10 63.5 association  | discrimination

Pascal VOC split 2

Shot
nAP50
download
1 30.5 association  | discrimination
2 35.1 association  | discrimination
3 40.3 association  | discrimination
5 42.9 association  | discrimination
10 48.3 association  | discrimination

Pascal VOC split 3

Shot
nAP50
download
1 45.7 association  | discrimination
2 49.4 association  | discrimination
3 49.4 association  | discrimination
5 55.1 association  | discrimination
10 59.3 association  | discrimination
Owner
Cao Yuhang
Cao Yuhang
Tutorial page of the Climate Hack, the greatest hackathon ever

Tutorial page of the Climate Hack, the greatest hackathon ever

UCL Artificial Intelligence Society 12 Jul 02, 2022
GuideDog is an AI/ML-based mobile app designed to assist the lives of the visually impaired, 100% voice-controlled

Guidedog Authors: Kyuhee Jo, Steven Gunarso, Jacky Wang, Raghav Sharma GuideDog is an AI/ML-based mobile app designed to assist the lives of the visua

Kyuhee Jo 5 Nov 24, 2021
UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering

UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering This repository holds all the code and data for our recent work on

Mohamed El Banani 118 Dec 06, 2022
This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

4 Aug 02, 2022
A collection of Jupyter notebooks to play with NVIDIA's StyleGAN3 and OpenAI's CLIP for a text-based guided image generation.

StyleGAN3 CLIP-based guidance StyleGAN3 + CLIP StyleGAN3 + inversion + CLIP This repo is a collection of Jupyter notebooks made to easily play with St

Eugenio Herrera 176 Dec 30, 2022
An educational AI robot based on NVIDIA Jetson Nano.

JetBot Looking for a quick way to get started with JetBot? Many third party kits are now available! JetBot is an open-source robot based on NVIDIA Jet

NVIDIA AI IOT 2.6k Dec 29, 2022
MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system

MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system Getting started To start working on this assignment, you should

2 Aug 06, 2022
Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Fastformer-PyTorch Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Usage : import t

Hong-Jia Chen 126 Dec 06, 2022
City Surfaces: City-scale Semantic Segmentation of Sidewalk Surfaces

City Surfaces: City-scale Semantic Segmentation of Sidewalk Surfaces Paper Temporary GitHub page for City Surfaces paper. More soon! While designing s

14 Nov 10, 2022
AAAI 2022 paper - Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction

AT-BMC Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction (AAAI 2022) Paper Prerequisites Install pac

16 Nov 26, 2022
RSC-Net: 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos

RSC-Net: 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos Implementation for "3D Human Pose, Shape and Texture from Low-Resoluti

XiangyuXu 42 Nov 10, 2022
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Pranav 39 Nov 21, 2022
This is the winning solution of the Endocv-2021 grand challange.

Endocv2021-winner [Paper] This is the winning solution of the Endocv-2021 grand challange. Dependencies pytorch # tested with 1.7 and 1.8 torchvision

Vajira Thambawita 14 Dec 03, 2022
Official Implementation of DE-DETR and DELA-DETR in "Towards Data-Efficient Detection Transformers"

DE-DETRs By Wen Wang, Jing Zhang, Yang Cao, Yongliang Shen, and Dacheng Tao This repository is an official implementation of DE-DETR and DELA-DETR in

Wen Wang 61 Dec 12, 2022
Official implementation of EfficientPose

EfficientPose This is the official implementation of EfficientPose. We based our work on the Keras EfficientDet implementation xuannianz/EfficientDet

2 May 17, 2022
Deep Learning (with PyTorch)

Deep Learning (with PyTorch) This notebook repository now has a companion website, where all the course material can be found in video and textual for

Alfredo Canziani 6.2k Jan 07, 2023
GPU-Accelerated Deep Learning Library in Python

Hebel GPU-Accelerated Deep Learning Library in Python Hebel is a library for deep learning with neural networks in Python using GPU acceleration with

Hannes Bretschneider 1.2k Dec 21, 2022
ADB-IP-ROTATION - Use your mobile phone to gain a temporary IP address using ADB and data tethering

ADB IP ROTATE This an Python script based on Android Debug Bridge (adb) shell sc

Dor Bismuth 2 Jul 12, 2022
Official Implementation of DDOD (Disentangle your Dense Object Detector), ACM MM2021

Disentangle Your Dense Object Detector This repo contains the supported code and configuration files to reproduce object detection results of Disentan

loveSnowBest 51 Jan 07, 2023