Joint Implicit Image Function for Guided Depth Super-Resolution

This repository contains the code for:

Joint Implicit Image Function for Guided Depth Super-Resolution
Jiaxiang Tang, Xiaokang Chen, Gang Zeng
ACM MM 2021

Installation

Environments:

• Python >= 3.6
• PyTorch >= 1.6.0
• tensorboardX
• tqdm, opencv-python, Pillow
• NVIDIA apex (python-only build is ok.)

Data preparation

Please see data/prepare_data.md for the details.

Training

You can use the provided scripts (scripts/train*) to train models.

For example:

# train JIIF with scale = 8 on the NYU dataset.
OMP_NUM_THREADS=8 CUDA_VISIBLE_DEVICES=2 python main.py \
    --name jiif_8 --model JIIF --scale 8 \
    --sample_q 30720 --input_size 256 --train_batch 1 \
    --epoch 200 --eval_interval 10 \
    --lr 0.0001 --lr_step 60 --lr_gamma 0.2

Testing

To test the performance of the models on difference datasets, you can use the provided scripts (scripts/test*).

For example:

# test the best checkpoint on MiddleBury dataest with scale = 8
OMP_NUM_THREADS=8 CUDA_VISIBLE_DEVICES=1 python main.py \
    --test --checkpoint best \
    --name jiif_8 --model JIIF \
    --dataset Middlebury --scale 8 --data_root ./data/depth_enhance/01_Middlebury_Dataset

Pretrained models and Reproducing

We provide the pretrained models here.

To test the performance of the pretrained models, please download the corresponding models and put them under pretrained folder. Then you can use scripts/test_jiif_pretrained.sh and scripts/test_denoise_jiif_pretrained.sh to reproduce the results reported in our paper.

Citation

If you find the code useful for your research, please use the following BibTeX entry:

@article{tang2021joint,
    title        = {Joint Implicit Image Function for Guided Depth Super-Resolution},
    author       = {Jiaxiang Tang, Xiaokang Chen, Gang Zeng},
    year         = 2021,
    journal      = {arXiv preprint arXiv:2107.08717}
}

Acknowledgment

The model implementation is based on liif.