Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

Related tags

Deep LearningNLOS-OT
Overview

NLOS-OT

Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

Description

In this repository, we release the NLOS-OT code in Pytorch as well as the passive NLOS imaging dataset NLOS-Passive.

  • Problem statement: Passive NLOS imaging

  • NLOS-OT architecture

  • The reconstruction results of NLOS-OT trained by specific dataset without partial occluder

  • The generalization results of NLOS-OT trained by dataset only from STL-10 with unknown partial occluder

Installation

  1. install required packages

  2. clone the repo

Prepare Data

  1. Download dataset

You can download each group in NLOS-Passive through the link below. Please note that a compressed package (.zip or .z01+.zip) represents a group of measured data.

link:https://pan.baidu.com/s/19Q48BWm1aJQhIt6BF9z-uQ

code:j3p2

If the link fails, please feel free to contact me.

  1. Organize the files structure of the dataset

Demo / Evaluate

Before that, you should have installed the required packages and organized the data set according to the appropriate file structure.

  1. Download pretrained pth

  2. run the test.py

Train

Before that, you should have installed the required packages and organized the data set according to the appropriate file structure.

Citation

If you find our work and code helpful, please consider cite:

We thank the following great works:

  • DeblurGAN, pix2pix: Our code is based on the framework provided by the two repos.

  • IntroVAE: The encoder and decoder in NLOS-OT are based on IntroVAE.

  • AE-OT, AEOT-GAN: The idea of using OT to complete passive NLOS imaging tasks in NLOS-OT comes from the two works.

If you find them helpful, please cite:

@inproceedings{kupynDeblurGANBlindMotion2018,
	title = {{DeblurGAN}: {Blind} {Motion} {Deblurring} {Using} {Conditional} {Adversarial} {Networks}},
	booktitle = {2018 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
	author = {Kupyn, Orest and Budzan, Volodymyr and Mykhailych, Mykola and Mishkin, Dmytro and Matas, Jiri},
	year = {2018},
}

@inproceedings{isolaImagetoimageTranslationConditional2017,
	title = {Image-to-image translation with conditional adversarial networks},
	booktitle = {2017 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
	publisher = {IEEE},
	author = {Isola, Phillip and Zhu, Jun-Yan and Zhou, Tinghui and Efros, Alexei A.},
	year = {2017},
	pages = {5967--5976},
}

@inproceedings{huang_introvae_2018,
	title = {{IntroVAE}: {Introspective} {Variational} {Autoencoders} for {Photographic} {Image} {Synthesis}},
	shorttitle = {{IntroVAE}},
	urldate = {2020-07-14},
	booktitle = {Advances in neural information processing systems},
	author = {Huang, Huaibo and Li, Zhihang and He, Ran and Sun, Zhenan and Tan, Tieniu},
	month = oct,
	year = {2018}
}

@article{an_ae-ot-gan_2020,
	title = {{AE}-{OT}-{GAN}: {Training} {Gans} from {Data} {Specific} {Latent} {Distribution}},
	shorttitle = {Ae-{Ot}-{Gan}},
	journal = {arXiv},
	author = {An, Dongsheng and Guo, Yang and Zhang, Min and Qi, Xin and Lei, Na and Yau, Shing-Tung and Gu, Xianfeng},
	year = {2020}
}

@inproceedings{an_ae-ot_2020,
	title = {{AE}-{OT}: {A} {NEW} {GENERATIVE} {MODEL} {BASED} {ON} {EX}- {TENDED} {SEMI}-{DISCRETE} {OPTIMAL} {TRANSPORT}},
	language = {en},
	author = {An, Dongsheng and Guo, Yang and Lei, Na and Luo, Zhongxuan and Yau, Shing-Tung and Gu, Xianfeng},
	year = {2020},
	pages = {19},
}
Owner
Ruixu Geng(耿瑞旭)
Undergraduate 2015 - 2019 (Expected), Information and Communication Engineering, UESTC
Ruixu Geng(耿瑞旭)
Anime Face Detector using mmdet and mmpose

Anime Face Detector This is an anime face detector using mmdetection and mmpose. (To avoid copyright issues, I use generated images by the TADNE model

198 Jan 07, 2023
Progressive Domain Adaptation for Object Detection

Progressive Domain Adaptation for Object Detection Implementation of our paper Progressive Domain Adaptation for Object Detection, based on pytorch-fa

96 Nov 25, 2022
[CVPR2021 Oral] End-to-End Video Instance Segmentation with Transformers

VisTR: End-to-End Video Instance Segmentation with Transformers This is the official implementation of the VisTR paper: Installation We provide instru

Yuqing Wang 687 Jan 07, 2023
A geometric deep learning pipeline for predicting protein interface contacts.

A geometric deep learning pipeline for predicting protein interface contacts.

44 Dec 30, 2022
An Industrial Grade Federated Learning Framework

DOC | Quick Start | 中文 FATE (Federated AI Technology Enabler) is an open-source project initiated by Webank's AI Department to provide a secure comput

Federated AI Ecosystem 4.8k Jan 09, 2023
Code for ACM MM2021 paper "Complementary Trilateral Decoder for Fast and Accurate Salient Object Detection"

CTDNet The PyTorch code for ACM MM2021 paper "Complementary Trilateral Decoder for Fast and Accurate Salient Object Detection" Requirements Python 3.6

CVTEAM 28 Oct 20, 2022
Differentiable Abundance Matching With Python

shamnet Differentiable Stellar Population Synthesis Installation You can install shamnet with pip. Installation dependencies are numpy, jax, corrfunc,

5 Dec 17, 2021
A Context-aware Visual Attention-based training pipeline for Object Detection from a Webpage screenshot!

CoVA: Context-aware Visual Attention for Webpage Information Extraction Abstract Webpage information extraction (WIE) is an important step to create k

Keval Morabia 41 Jan 01, 2023
Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo This repository includes the source code for our CVPR 2021 paper on multi-view mult

Jiahao Lin 66 Jan 04, 2023
Main repository for the HackBio'2021 Virtual Internship Experience for #Team-Greider ❤️

Hello 🤟 #Team-Greider The team of 20 people for HackBio'2021 Virtual Bioinformatics Internship 💝 🖨️ 👨‍💻 HackBio: https://thehackbio.com 💬 Ask us

Siddhant Sharma 7 Oct 20, 2022
Training DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)

Training DALL-E with volunteers from all over the Internet This repository is a part of the NeurIPS 2021 demonstration "Training Transformers Together

<a href=[email protected]"> 19 Dec 13, 2022
This MVP data web app uses the Streamlit framework and Facebook's Prophet forecasting package to generate a dynamic forecast from your own data.

📈 Automated Time Series Forecasting Background: This MVP data web app uses the Streamlit framework and Facebook's Prophet forecasting package to gene

Zach Renwick 42 Jan 04, 2023
Official implementation of Unfolded Deep Kernel Estimation for Blind Image Super-resolution.

Unfolded Deep Kernel Estimation for Blind Image Super-resolution Hongyi Zheng, Hongwei Yong, Lei Zhang, "Unfolded Deep Kernel Estimation for Blind Ima

Z80 15 Dec 26, 2022
Simple Pixelbot for Diablo 2 Resurrected written in python and opencv.

Simple Pixelbot for Diablo 2 Resurrected written in python and opencv. Obviously only use it in offline mode as it is against the TOS of Blizzard to use it in online mode!

468 Jan 03, 2023
Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"

GPT-GNN: Generative Pre-Training of Graph Neural Networks GPT-GNN is a pre-training framework to initialize GNNs by generative pre-training. It can be

Ziniu Hu 346 Dec 19, 2022
Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification Official Implementation for the pape

Anh M. Nguyen 36 Dec 28, 2022
Training a deep learning model on the noisy CIFAR dataset

Training-a-deep-learning-model-on-the-noisy-CIFAR-dataset This repository contai

1 Jun 14, 2022
Implementation of paper "Self-supervised Learning on Graphs:Deep Insights and New Directions"

SelfTask-GNN A PyTorch implementation of "Self-supervised Learning on Graphs: Deep Insights and New Directions". [paper] In this paper, we first deepe

Wei Jin 85 Oct 13, 2022
Python implementation of Bayesian optimization over permutation spaces.

Bayesian Optimization over Permutation Spaces This repository contains the source code and the resources related to the paper "Bayesian Optimization o

Aryan Deshwal 9 Dec 23, 2022
Builds a LoRa radio frequency fingerprint identification (RFFI) system based on deep learning techiniques

This project builds a LoRa radio frequency fingerprint identification (RFFI) system based on deep learning techiniques.

20 Dec 30, 2022