This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Last update: May 03, 2022

Related tags

Deep Learning ObjProp

Overview

ObjProp

Introduction

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Installation

This repo is built using mmdetection. To install the dependencies, first clone the repository locally:

git clone https://github.com/anirudh-chakravarthy/objprop.git

Then, install PyTorch 1.1.0, torchvision 0.3.0, mmcv 0.2.12:

conda install pytorch==1.1.0 torchvision==0.3.0 -c pytorch
pip install mmcv==0.2.12

Then, install the CocoAPI for YouTube-VIS

conda install cython scipy
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"

Training

First, download and prepare the YouTube-VIS dataset using the following instructions.

To train ObjProp, run the following command:

python3 tools/train.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py

In order to change the arguments such as dataset directory, learning rate, number of GPUs, etc, refer to the following configuration file configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py.

Inference

To perform inference using ObjProp, run the following command:

python3 tools/test_video.py configs/masktrack_rcnn_r50_fpn_1x_youtubevos_objprop.py [MODEL_PATH] --out [OUTPUT_PATH.json] --eval segm

A JSON file with the inference results will be saved at OUTPUT_PATH.json. To evaluate the performance, submit the result file to the evaluation server.

License

ObjProp is released under the Apache 2.0 license.

Citation

@article{Chakravarthy2021ObjProp,
  author = {Anirudh S Chakravarthy and Won-Dong Jang and Zudi Lin and Donglai Wei and Song Bai and Hanspeter Pfister},  
  title = {Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation},
  journal = {CoRR},
  volume = {abs/2111.07529},
  year = {2021},
  url = {https://arxiv.org/abs/2111.07529}
}

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

Related tags

Overview

ObjProp

Introduction

Installation

Training

Inference

License

Citation

Owner

Anirudh S Chakravarthy

Detecting Blurred Ground-based Sky/Cloud Images

Study of human inductive biases in CNNs and Transformers.

This is an open source library implementing hyperbox-based machine learning algorithms

Stock-Prediction - prediction of stock market movements using sentiment analysis and deep learning.

PyTorch implementation of the paper Deep Networks from the Principle of Rate Reduction

OpenMMLab Computer Vision Foundation

This repository contains demos I made with the Transformers library by HuggingFace.

Interactive dimensionality reduction for large datasets

The Ludii general game system, developed as part of the ERC-funded Digital Ludeme Project.

Implementations of the algorithms in the paper Approximative Algorithms for Multi-Marginal Optimal Transport and Free-Support Wasserstein Barycenters

Code for Subgraph Federated Learning with Missing Neighbor Generation (NeurIPS 2021)

AITUS - An atomatic notr maker for CYTUS

Python package to generate image embeddings with CLIP without PyTorch/TensorFlow

Code artifacts for the submission "Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems"

TigerLily: Finding drug interactions in silico with the Graph.

This is an official implementation for the WTW Dataset in "Parsing Table Structures in the Wild " on table detection and table structure recognition.

Fine-grained Control of Image Caption Generation with Abstract Scene Graphs

U^2-Net - Portrait matting This repository explores possibilities of using the original u^2-net model for portrait matting.

A novel benchmark dataset for Monocular Layout prediction

Learning Dense Representations of Phrases at Scale (Lee et al., 2020)