Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Last update: Dec 06, 2022

Overview

In-the-Wild Single Camera 3D Reconstruction
Through Moving Water Surfaces

This is the PyTorch implementation for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Project Page | Paper | Supplemental Material

In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces
Jinhui Xiong, Wolfgang Heidrich
KAUST
ICCV 2021 (Oral)

We propose a differentiable framework to estimate underwater scene geometry along with the time-varying water surface. The inputs to our model are a video sequence captured by a fixed camera. Dense correspondence from each frame to a world reference frame (selected from the input sequences) is pre-computed, ensuring the reconstruction is performed in a unified coordinate system. We feed the flow fields, together with initialized water surfaces and scene geometry (all are initialized as planar surfaces), into the framework, which incorporates ray casting, Snell’s law and multi-view triangulation. The gradients of the specially designed losses with respect to water surfaces and scene geometry are back-propagated, and all parameters are simultaneously optimized. The final result is a quality reconstruction of the underwater scene, along with an estimate of the time-varying water-air interface. The data shown here was captured in a public fountain environment.

Prerequisite

The code was tested with python>=3.7 & PyTorch>=1.3 & cuda>=10.0 on Nvidia RTX 2080 Ti
Minor change on the code if there is compatibility issue. It needs around 10 GB GPU memory.

Setup

conda create -n moving_water python=3.7
conda activate moving_water

conda install pytorch torchvision -c pytorch
conda install -c conda-forge opencv scikit-image
conda install -c anaconda scipy

Run the code

Please go to example folder, download the cached coefficient matrices (there are three matrices for each example) and execute:

python3 run.py

Citation

@inproceedings{xiong2021inthewild,
  title={In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces},
  author={Jinhui Xiong and Wolfgang Heidrich},
  year={2021},
  booktitle={ICCV}
}

Contact

Please contact Jinhui Xiong [email protected] if you have any question or comment.

Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Related tags

Overview

In-the-Wild Single Camera 3D Reconstruction
Through Moving Water Surfaces

Project Page | Paper | Supplemental Material

Prerequisite

Setup

Run the code

Citation

Contact

Owner

Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving

One Million Scenes for Autonomous Driving

Official code for "Mean Shift for Self-Supervised Learning"

A simple, fully convolutional model for real-time instance segmentation.

A PyTorch Image-Classification With AlexNet And ResNet50.

A robust camera and Lidar fusion based velocity estimator to undistort the pointcloud.

Building blocks for uncertainty-aware cycle consistency presented at NeurIPS'21.

Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer)

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

[3DV 2021] Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation

A curated list of awesome Deep Learning tutorials, projects and communities.

Code and Datasets from the paper "Self-supervised contrastive learning for volcanic unrest detection from InSAR data"

LRBoost is a scikit-learn compatible approach to performing linear residual based stacking/boosting.

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

📚 Papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks.

Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

I-BERT: Integer-only BERT Quantization

Sharpened cosine similarity torch - A Sharpened Cosine Similarity layer for PyTorch

DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks