Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Last update: Dec 06, 2022

Overview

In-the-Wild Single Camera 3D Reconstruction
Through Moving Water Surfaces

This is the PyTorch implementation for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Project Page | Paper | Supplemental Material

In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces
Jinhui Xiong, Wolfgang Heidrich
KAUST
ICCV 2021 (Oral)

We propose a differentiable framework to estimate underwater scene geometry along with the time-varying water surface. The inputs to our model are a video sequence captured by a fixed camera. Dense correspondence from each frame to a world reference frame (selected from the input sequences) is pre-computed, ensuring the reconstruction is performed in a unified coordinate system. We feed the flow fields, together with initialized water surfaces and scene geometry (all are initialized as planar surfaces), into the framework, which incorporates ray casting, Snell’s law and multi-view triangulation. The gradients of the specially designed losses with respect to water surfaces and scene geometry are back-propagated, and all parameters are simultaneously optimized. The final result is a quality reconstruction of the underwater scene, along with an estimate of the time-varying water-air interface. The data shown here was captured in a public fountain environment.

Prerequisite

The code was tested with python>=3.7 & PyTorch>=1.3 & cuda>=10.0 on Nvidia RTX 2080 Ti
Minor change on the code if there is compatibility issue. It needs around 10 GB GPU memory.

Setup

conda create -n moving_water python=3.7
conda activate moving_water

conda install pytorch torchvision -c pytorch
conda install -c conda-forge opencv scikit-image
conda install -c anaconda scipy

Run the code

Please go to example folder, download the cached coefficient matrices (there are three matrices for each example) and execute:

python3 run.py

Citation

@inproceedings{xiong2021inthewild,
  title={In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces},
  author={Jinhui Xiong and Wolfgang Heidrich},
  year={2021},
  booktitle={ICCV}
}

Contact

Please contact Jinhui Xiong [email protected] if you have any question or comment.

Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Related tags

Overview

In-the-Wild Single Camera 3D Reconstruction
Through Moving Water Surfaces

Project Page | Paper | Supplemental Material

Prerequisite

Setup

Run the code

Citation

Contact

Owner

Codes for TIM2021 paper "Anchor-Based Spatio-Temporal Attention 3-D Convolutional Networks for Dynamic 3-D Point Cloud Sequences"

An Official Repo of CVPR '20 "MSeg: A Composite Dataset for Multi-Domain Segmentation"

Use Python, OpenCV, and MediaPipe to control a keyboard with facial gestures

Deep Reinforcement Learning with pytorch & visdom

PyTorch Code for NeurIPS 2021 paper Anti-Backdoor Learning: Training Clean Models on Poisoned Data.

Code for our paper "Graph Pre-training for AMR Parsing and Generation" in ACL2022

KaziText is a tool for modelling common human errors.

LEDNet: A Lightweight Encoder-Decoder Network for Real-time Semantic Segmentation

Kaggle G2Net Gravitational Wave Detection : 2nd place solution

This repo contains the code and data used in the paper "Wizard of Search Engine: Access to Information Through Conversations with Search Engines"

Simple data balancing baselines for worst-group-accuracy benchmarks.

A chemical analysis of lipophilicities & molecule drawings including ML

source code the paper Fast and Robust Iterative Closet Point.

Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN

EEGEyeNet is benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty

U-Net Brain Tumor Segmentation

List of papers, code and experiments using deep learning for time series forecasting

Latent Network Models to Account for Noisy, Multiply-Reported Social Network Data

Convolutional Neural Network to detect deforestation in the Amazon Rainforest

PyTorch for Semantic Segmentation