Caffe implementation for Hu et al. Segmentation for Natural Language Expressions

Overview

Segmentation from Natural Language Expressions

This repository contains the Caffe reimplementation of the following paper:

  • R. Hu, M. Rohrbach, T. Darrell, Segmentation from Natural Language Expressions. in arXiv:1603.06180, 2016. (PDF)
@article{hu2016segmentation,
  title={Segmentation from Natural Language Expressions},
  author={Hu, Ronghang and Rohrbach, Marcus and Darrell, Trevor},
  journal={arXiv preprint arXiv:1603.06180},
  year={2016}
}

Project Page: http://ronghanghu.com/text_objseg

Installation

  1. Install Caffe following the instructions here.
  2. Download this repository or clone with Git, and then cd into the root directory of the repository.

Training and evaluation on ReferIt Dataset

Download dataset and VGG network

Download ReferIt dataset:

./referit/referit-dataset/download_referit_dataset.sh

Download the caffemodel for VGG-16 network parameters trained on ImageNET 1000 classes.

Training

You may need to add the repository root directory to Python's module path:

export PYTHONPATH=/path/to/text_objseg_caffe/:$PYTHONPATH

Build training batches for bounding boxes:

python referit/build_training_batches_det.py

Build training batches for segmentation:

python referit/build_training_batches_seg.py

Configure the config.py file in the directory det_model and train the language-based bounding box localization model:

python det_model/train_det_model.py

Configure the config.py file in the directory seg_low_res_model and train the low resolution language-based segmentation model (from the previous bounding box localization model):

python seg_low_res_model/train_low_res_model.py

Configure the config.py file in the directory seg_model and train the high resolution language-based segmentation model (from the previous low resolution segmentation model):

python seg_model/train_seg_model.py

Evaluation

You may need to add the repository root directory to Python's module path:

export PYTHONPATH=path/to/text_objseg_caffe:$PYTHONPATH

Configure the test_config.py file in the directory seg_model and run evaluation for the high resolution language-based segmentation model:

python seg_model/test_seg_model.py

This should reproduce the results in the paper. You may also evaluate the language-based bounding box localization model:

python det_model/test_det_model.py

The results can be compared to this paper.

Demo

There is a demo that you can try! Run the demo in ./demo/text_objseg_demo.ipynb with Jupyter Notebook (IPython Notebook).

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022
Face Recognize System on camera AI OAK1

FRS on OAK1 Face Recognize System on camera OAK1 This project contains our work that deploy on camera OAK1 Features Anti-Spoofing Face detection Face

Tran Anh Tuan 6 Aug 08, 2022
PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement.

DECOR-GAN PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement, Zhiqin Chen, Vladimir G. Kim, Matthew Fish

Zhiqin Chen 72 Dec 31, 2022
Weighted QMIX: Expanding Monotonic Value Function Factorisation

This repo contains the cleaned-up code that was used in "Weighted QMIX: Expanding Monotonic Value Function Factorisation"

whirl 82 Dec 29, 2022
Code To Tune or Not To Tune? Zero-shot Models for Legal Case Entailment.

COLIEE 2021 - task 2: Legal Case Entailment This repository contains the code to reproduce NeuralMind's submissions to COLIEE 2021 presented in the pa

NeuralMind 13 Dec 16, 2022
FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection

FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection arXi

59 Nov 29, 2022
QuadTree Attention for Vision Transformers (ICLR2022)

This repository contains codes for quadtree attention. This repo contains codes for feature matching, image classficiation, object detection and seman

tangshitao 222 Dec 28, 2022
Melanoma Skin Cancer Detection using Convolutional Neural Networks and Transfer LearningπŸ•΅πŸ»β€β™‚οΈ

This is a Kaggle competition in which we have to identify if the given lesion image is malignant or not for Melanoma which is a type of skin cancer.

Vipul Shinde 1 Jan 27, 2022
A PyTorch implementation of QANet.

QANet-pytorch NOTICE I'm very busy these months. I'll return to this repo in about 10 days. Introduction An implementation of QANet with PyTorch. Any

H. Z. 343 Nov 03, 2022
Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biom

Phil Wang 59 Nov 24, 2022
Deploy recommendation engines with Edge Computing

RecoEdge: Bringing Recommendations to the Edge A one stop solution to build your recommendation models, train them and, deploy them in a privacy prese

NimbleEdge 131 Jan 02, 2023
Some methods for comparing network representations in deep learning and neuroscience.

Generalized Shape Metrics on Neural Representations In neuroscience and in deep learning, quantifying the (dis)similarity of neural representations ac

Alex Williams 45 Dec 27, 2022
PyTorch implementation of a collections of scalable Video Transformer Benchmarks.

PyTorch implementation of Video Transformer Benchmarks This repository is mainly built upon Pytorch and Pytorch-Lightning. We wish to maintain a colle

Xin Ma 156 Jan 08, 2023
Compute descriptors for 3D point cloud registration using a multi scale sparse voxel architecture

MS-SVConv : 3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning Compute features for 3D point cloud registration

42 Jul 25, 2022
Learning Continuous Signed Distance Functions for Shape Representation

DeepSDF This is an implementation of the CVPR '19 paper "DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation" by Park et a

Meta Research 1.1k Jan 01, 2023
E2e music remastering system - End-to-end Music Remastering System Using Self-supervised and Adversarial Training

End-to-end Music Remastering System This repository includes source code and pre

Junghyun (Tony) Koo 37 Dec 15, 2022
μ‹œκ° μž₯애인을 μœ„ν•œ 슀마트 μ§€νŒ‘μ΄μ— ν™œμš©λ  λ”₯λŸ¬λ‹ λͺ¨λΈ (DL Model Repo)

SmartCane-DL-Model Smart Cane using semantic segmentation μ°Έκ³ ν•œ Github repositoy πŸ”— https://github.com/JunHyeok96/Road-Segmentation.git 데이터셋 πŸ”— https://

λ°˜λ“œμ‹œ μ‘Έμ—…ν•œλ‹€ (Team Just Graduate) 4 Dec 03, 2021
[ICRA 2022] CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation

This is the official implementation of our paper: Bowen Wen, Wenzhao Lian, Kostas Bekris, and Stefan Schaal. "CaTGrasp: Learning Category-Level Task-R

Bowen Wen 199 Jan 04, 2023
A hand tracking demo made with mediapipe where you can control lights with pinching your fingers and moving your hand up/down.

HandTrackingBrightnessControl A hand tracking demo made with mediapipe where you can control lights with pinching your fingers and moving your hand up

Teemu Laurila 19 Feb 12, 2022
Clinica is a software platform for clinical research studies involving patients with neurological and psychiatric diseases and the acquisition of multimodal data

Clinica Software platform for clinical neuroimaging studies Homepage | Documentation | Paper | Forum | See also: AD-ML, AD-DL ClinicaDL About The Proj

ARAMIS Lab 165 Dec 29, 2022