This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Last update: May 03, 2022

Related tags

Deep Learning ObjProp

Overview

ObjProp

Introduction

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Installation

This repo is built using mmdetection. To install the dependencies, first clone the repository locally:

git clone https://github.com/anirudh-chakravarthy/objprop.git

Then, install PyTorch 1.1.0, torchvision 0.3.0, mmcv 0.2.12:

conda install pytorch==1.1.0 torchvision==0.3.0 -c pytorch
pip install mmcv==0.2.12

Then, install the CocoAPI for YouTube-VIS

conda install cython scipy
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"

Training

First, download and prepare the YouTube-VIS dataset using the following instructions.

To train ObjProp, run the following command:

python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py

In order to change the arguments such as dataset directory, learning rate, number of GPUs, etc, refer to the following configuration file configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py.

Inference

To perform inference using ObjProp, run the following command:

python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py [MODEL_PATH] --out [OUTPUT_PATH.json] --eval segm

A JSON file with the inference results will be saved at OUTPUT_PATH.json. To evaluate the performance, submit the result file to the evaluation server.

License

ObjProp is released under the Apache 2.0 license.

Citation

@article{Chakravarthy2021ObjProp,
  author = {Anirudh S Chakravarthy and Won-Dong Jang and Zudi Lin and Donglai Wei and Song Bai and Hanspeter Pfister},  
  title = {Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation},
  journal = {CoRR},
  volume = {abs/2111.07529},
  year = {2021},
  url = {https://arxiv.org/abs/2111.07529}
}

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Related tags

Overview

ObjProp

Introduction

Installation

Training

Inference

License

Citation

Owner

Anirudh S Chakravarthy

Towards Fine-Grained Reasoning for Fake News Detection

Improving Calibration for Long-Tailed Recognition (CVPR2021)

Code for paper Novel View Synthesis via Depth-guided Skip Connections

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit

Annotate datasets with a semi-trained or fully trained YOLOv5 model

Official implementation for Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

This repository introduces a short project about Transfer Learning for Classification of MRI Images.

Low-code/No-code approach for deep learning inference on devices

Official repository for Natural Image Matting via Guided Contextual Attention

Ontologysim: a Owlready2 library for applied production simulation

Posterior temperature optimized Bayesian models for inverse problems in medical imaging

Code accompanying our NeurIPS 2021 traffic4cast challenge

Spatial Temporal Graph Convolutional Networks (ST-GCN) for Skeleton-Based Action Recognition in PyTorch

Exploring whether attention is necessary for vision transformers

Reinforcement learning models in ViZDoom environment

The devkit of the nuScenes dataset.

Joint Unsupervised Learning (JULE) of Deep Representations and Image Clusters.

Pytorch implementation of Integrating Tree Path in Transformer for Code Representation

FANet - Real-time Semantic Segmentation with Fast Attention