Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

Last update: Dec 26, 2022

Overview

UniFuse (RAL+ICRA2021)

Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation, arXiv, Demo

Preparation

Installation

Environments

python 3.6
Pytorch >= 1.0.0
CUDA >= 9.0

Install requirements

pip install -r requirements.txt

Datasets

Please download the preferred datasets, i.e., Matterport3D, Stanford2D3D, 3D60 and PanoSUNCG. For Matterport3D, please preprocess it following M3D-README.md.

Training

UniFuse on Matterport3D

python train.py --data_path $DATA_PATH \
-dataset matterport3d \
--model_name Matterport3D_UniFuse \
--batch_size 6 \
--num_epochs 100 \
--height 512 \
--width 1024 \
--imagenet_pretrained \
--net UniFuse

Equirectangular baseline on Matterport3D

python train.py --data_path $DATA_PATH \
-dataset matterport3d \
--model_name Matterport3D_Equi \
--batch_size 6 \
--num_epochs 100 \
--height 512 \
--width 1024 \
--imagenet_pretrained \
--net Equi

It is similar for other datasets.

Evaluation

Pre-trained models

The pre-trained models of UniFuse for 4 datasets are available, Matterport3D, Stanford2D3D, 3D60 and PanoSUNCG.

Test on a pre-trained model

python evaluate.py  --data_path $DATA_PATH --dataset matterport3d --load_weights_folder $MODEL_PATH

Citation

Please cite our paper if you find our work useful in your research.

@article{jiang2021unifuse,
      title={UniFuse: Unidirectional Fusion for 360$^{\circ}$ Panorama Depth Estimation}, 
      author={Hualie Jiang and Zhe Sheng and Siyu Zhu and Zilong Dong and Rui Huang},
	  journal={IEEE Robotics and Automation Letters},
	  year={2021},
	  publisher={IEEE}
}

Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

Related tags

Overview

UniFuse (RAL+ICRA2021)

Preparation

Installation

Datasets

Training

UniFuse on Matterport3D

Equirectangular baseline on Matterport3D

Evaluation

Pre-trained models

Test on a pre-trained model

Citation

Owner

Alibaba

Temporally Efficient Vision Transformer for Video Instance Segmentation, CVPR 2022, Oral

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

Autonomous Driving on Curvy Roads without Reliance on Frenet Frame: A Cartesian-based Trajectory Planning Method

Implementation of Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning

This repository contains project created during the Data Challenge module at London School of Hygiene & Tropical Medicine

Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch

Kinetics-Data-Preprocessing

MVSDF - Learning Signed Distance Field for Multi-view Surface Reconstruction

The Ludii general game system, developed as part of the ERC-funded Digital Ludeme Project.

SegTransVAE: Hybrid CNN - Transformer with Regularization for medical image segmentation

Learning trajectory representations using self-supervision and programmatic supervision.

EEGEyeNet is benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty

Self-supervised learning algorithms provide a way to train Deep Neural Networks in an unsupervised way using contrastive losses

SplineConv implementation for Paddle.

Code release for paper: The Boombox: Visual Reconstruction from Acoustic Vibrations

Improving Query Representations for DenseRetrieval with Pseudo Relevance Feedback:A Reproducibility Study.

Permeability Prediction Via Multi Scale 3D CNN

Robustness via Cross-Domain Ensembles

Technical Analysis library in pandas for backtesting algotrading and quantitative analysis

FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.