Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

Related tags

Deep LearningNLOS-OT
Overview

NLOS-OT

Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

Description

In this repository, we release the NLOS-OT code in Pytorch as well as the passive NLOS imaging dataset NLOS-Passive.

  • Problem statement: Passive NLOS imaging

  • NLOS-OT architecture

  • The reconstruction results of NLOS-OT trained by specific dataset without partial occluder

  • The generalization results of NLOS-OT trained by dataset only from STL-10 with unknown partial occluder

Installation

  1. install required packages

  2. clone the repo

Prepare Data

  1. Download dataset

You can download each group in NLOS-Passive through the link below. Please note that a compressed package (.zip or .z01+.zip) represents a group of measured data.

link:https://pan.baidu.com/s/19Q48BWm1aJQhIt6BF9z-uQ

code:j3p2

If the link fails, please feel free to contact me.

  1. Organize the files structure of the dataset

Demo / Evaluate

Before that, you should have installed the required packages and organized the data set according to the appropriate file structure.

  1. Download pretrained pth

  2. run the test.py

Train

Before that, you should have installed the required packages and organized the data set according to the appropriate file structure.

Citation

If you find our work and code helpful, please consider cite:

We thank the following great works:

  • DeblurGAN, pix2pix: Our code is based on the framework provided by the two repos.

  • IntroVAE: The encoder and decoder in NLOS-OT are based on IntroVAE.

  • AE-OT, AEOT-GAN: The idea of using OT to complete passive NLOS imaging tasks in NLOS-OT comes from the two works.

If you find them helpful, please cite:

@inproceedings{kupynDeblurGANBlindMotion2018,
	title = {{DeblurGAN}: {Blind} {Motion} {Deblurring} {Using} {Conditional} {Adversarial} {Networks}},
	booktitle = {2018 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
	author = {Kupyn, Orest and Budzan, Volodymyr and Mykhailych, Mykola and Mishkin, Dmytro and Matas, Jiri},
	year = {2018},
}

@inproceedings{isolaImagetoimageTranslationConditional2017,
	title = {Image-to-image translation with conditional adversarial networks},
	booktitle = {2017 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
	publisher = {IEEE},
	author = {Isola, Phillip and Zhu, Jun-Yan and Zhou, Tinghui and Efros, Alexei A.},
	year = {2017},
	pages = {5967--5976},
}

@inproceedings{huang_introvae_2018,
	title = {{IntroVAE}: {Introspective} {Variational} {Autoencoders} for {Photographic} {Image} {Synthesis}},
	shorttitle = {{IntroVAE}},
	urldate = {2020-07-14},
	booktitle = {Advances in neural information processing systems},
	author = {Huang, Huaibo and Li, Zhihang and He, Ran and Sun, Zhenan and Tan, Tieniu},
	month = oct,
	year = {2018}
}

@article{an_ae-ot-gan_2020,
	title = {{AE}-{OT}-{GAN}: {Training} {Gans} from {Data} {Specific} {Latent} {Distribution}},
	shorttitle = {Ae-{Ot}-{Gan}},
	journal = {arXiv},
	author = {An, Dongsheng and Guo, Yang and Zhang, Min and Qi, Xin and Lei, Na and Yau, Shing-Tung and Gu, Xianfeng},
	year = {2020}
}

@inproceedings{an_ae-ot_2020,
	title = {{AE}-{OT}: {A} {NEW} {GENERATIVE} {MODEL} {BASED} {ON} {EX}- {TENDED} {SEMI}-{DISCRETE} {OPTIMAL} {TRANSPORT}},
	language = {en},
	author = {An, Dongsheng and Guo, Yang and Lei, Na and Luo, Zhongxuan and Yau, Shing-Tung and Gu, Xianfeng},
	year = {2020},
	pages = {19},
}
Owner
Ruixu Geng(耿瑞旭)
Undergraduate 2015 - 2019 (Expected), Information and Communication Engineering, UESTC
Ruixu Geng(耿瑞旭)
Pytorch implementation of Learning Rate Dropout.

Learning-Rate-Dropout Pytorch implementation of Learning Rate Dropout. Paper Link: https://arxiv.org/pdf/1912.00144.pdf Train ResNet-34 for Cifar10: r

42 Nov 25, 2022
Continuous Diffusion Graph Neural Network

We present Graph Neural Diffusion (GRAND) that approaches deep learning on graphs as a continuous diffusion process and treats Graph Neural Networks (GNNs) as discretisations of an underlying PDE.

Twitter Research 227 Jan 05, 2023
Implementation of the ALPHAMEPOL algorithm, presented in Unsupervised Reinforcement Learning in Multiple Environments.

ALPHAMEPOL This repository contains the implementation of the ALPHAMEPOL algorithm, presented in Unsupervised Reinforcement Learning in Multiple Envir

3 Dec 23, 2021
Auto White-Balance Correction for Mixed-Illuminant Scenes

Auto White-Balance Correction for Mixed-Illuminant Scenes Mahmoud Afifi, Marcus A. Brubaker, and Michael S. Brown York University Video Reference code

Mahmoud Afifi 47 Nov 26, 2022
Heterogeneous Temporal Graph Neural Network

Heterogeneous Temporal Graph Neural Network This repository contains the datasets and source code of HTGNN. run_mag.ipynb is the training and testing

15 Dec 22, 2022
Implementation of gaze tracking and demo

Predicting Customer Demand by Using Gaze Detecting and Object Tracking This project is the integration of gaze detecting and object tracking. Predict

2 Oct 20, 2022
Symbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis in JAX

SYMPAIS: Symbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis Overview | Installation | Documentation | Examples | Notebo

Yicheng Luo 4 Sep 13, 2022
Self-Supervised Learning for Domain Adaptation on Point-Clouds

Self-Supervised Learning for Domain Adaptation on Point-Clouds Introduction Self-supervised learning (SSL) allows to learn useful representations from

Idan Achituve 66 Dec 20, 2022
A demo of how to use JAX to create a simple gravity simulation

JAX Gravity This repo contains a demo of how to use JAX to create a simple gravity simulation. It uses JAX's experimental ode package to solve the dif

Cristian Garcia 16 Sep 22, 2022
A fuzzing framework for SMT solvers

yinyang A fuzzing framework for SMT solvers. Given a set of seed SMT formulas, yinyang generates mutant formulas to stress-test SMT solvers. yinyang c

Project Yin-Yang for SMT Solver Testing 145 Jan 04, 2023
Editing a Conditional Radiance Field

Editing Conditional Radiance Fields Project | Paper | Video | Demo Editing Conditional Radiance Fields Steven Liu, Xiuming Zhang, Zhoutong Zhang, Rich

Steven Liu 216 Dec 30, 2022
Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

Distributed Evolutionary Algorithms in Python 4.9k Jan 05, 2023
Chainer Implementation of Semantic Segmentation using Adversarial Networks

Semantic Segmentation using Adversarial Networks Requirements Chainer (1.23.0) Differences Use of FCN-VGG16 instead of Dilated8 as Segmentor. Caution

Taiki Oyama 99 Jun 28, 2022
Building a real-time environment using webcam frame division in OpenCV and classify cropped images using a fine-tuned vision transformers on hybryd datasets samples for facial emotion recognition.

Visual Transformer for Facial Emotion Recognition (FER) This project has the aim to build an efficient Visual Transformer for the Facial Emotion Recog

Mario Sessa 8 Dec 12, 2022
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Yi Wei 43 Dec 05, 2022
An attempt at the implementation of GLOM, Geoffrey Hinton's paper for emergent part-whole hierarchies from data

GLOM TensorFlow This Python package attempts to implement GLOM in TensorFlow, which allows advances made by several different groups transformers, neu

Rishit Dagli 32 Feb 21, 2022
A Pythonic library for Nvidia Codec.

A Pythonic library for Nvidia Codec. The project is still in active development; expect breaking changes. Why another Python library for Nvidia Codec?

Zesen Qian 12 Dec 27, 2022
Official Repository for our ICCV2021 paper: Continual Learning on Noisy Data Streams via Self-Purified Replay

Continual Learning on Noisy Data Streams via Self-Purified Replay This repository contains the official PyTorch implementation for our ICCV2021 paper.

Jinseo Jeong 22 Nov 23, 2022
An implementation of the AdaOPS (Adaptive Online Packing-based Search), which is an online POMDP Solver used to solve problems defined with the POMDPs.jl generative interface.

AdaOPS An implementation of the AdaOPS (Adaptive Online Packing-guided Search), which is an online POMDP Solver used to solve problems defined with th

9 Oct 05, 2022
This repo holds the code of TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation

TransFuse This repo holds the code of TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation Requirements Pytorch=1.6.0, 1.9.0 (=1.

Rayicer 93 Dec 19, 2022