Learnable Motion Coherence for Correspondence Pruning
Yuan Liu, Lingjie Liu, Cheng Lin, Zhen Dong, Wenping Wang
Project Page

Any questions or discussions are welcomed!

1. Requirements

Required packages are listed in requirements.txt.

The code is tested using Python-3.8.5 with PyTorch 1.7.1.

2. Compile extra modules

cd network/knn_search
python setup.py build_ext --inplace
cd ../pointnet2_ext
python setup.py build_ext --inplace
cd ../../utils/extend_utils
python build_extend_utils_cffi.py

According to your installation path of CUDA, you may need to revise the variables cuda_version in build_extend_utils_cffi.py.

1. Download the YFCC100M dataset and the SUN3D dataset from the OANet repository and the ScanNet dataset from here.

2. Download pretrained LMCNet models from here and SuperGlue/SuperPoint models from here. (geometry-only models are available at here.)

3. Unzip and arrange all files like the following.

data/
├── superpoint/
    └── superpoint_v1.pth
├── superglue/
    ├── superglue_indoor.pth
    └── superglue_outdoor.pth
├── model/
    ├── lmcnet_sift_indoor/
    ├── lmcnet_sift_outdoor/
    ├── lmcnet_sift_indoor_geom/
    ├── lmcnet_sift_outdoor_geom/
    └── lmcnet_spg_indoor/
├── yfcc100m/
├── sun3d_test/
├── sun3d_train/
├── scannet_dataset/
├── pairs/ # this was extracted from the dataset downloaded from OANet repository.
└── scannet_train_dataset/

Evaluate on the YFCC100M with SIFT descriptors and Nearest Neighborhood (NN) matcher:

python eval.py --name scannet --cfg configs/eval/lmcnet_sift_yfcc.yaml

Evaluate on the YFCC100M with SIFT descriptors and Nearest Neighborhood (NN) matcher using the geometry-only model:

python eval.py --name scannet --cfg configs/eval/lmcnet_sift_yfcc_geom.yaml

Evaluate on the SUN3D with SIFT descriptors and NN matcher:

python eval.py --name sun3d --cfg configs/eval/lmcnet_sift_sun3d.yaml

Evaluate on the ScanNet with SuperPoint descriptors and SuperGlue matcher:

python eval.py --name scannet --cfg configs/eval/lmcnet_spg_scannet.yaml

1. Generate training dataset for training on YFCC100M with SIFT descriptor and NN matcher.

python trainset_generate.py \
      --ext_cfg configs/detector/sift.yaml \
      --match_cfg configs/matcher/nn.yaml \
      --output data/yfcc_train_cache \
      --eig_name small_min \
      --prefix yfcc

2. Model training.

python train_model.py --cfg configs/lmcnet/lmcnet_sift_outdoor_train.yaml

@inproceedings{liu2021learnable,
  title={Learnable Motion Coherence for Correspondence Pruning},
  author={Liu, Yuan and Liu, Lingjie and Lin, Cheng and Dong, Zhen and Wang, Wenping},
  booktitle={CVPR}
  year={2021}
}

We have used codes from the following repositories, and we thank the authors for sharing their codes.

SuperGlue: https://github.com/magicleap/SuperGluePretrainedNetwork

OANet: https://github.com/zjhthu/OANet

KNN-CUDA: https://github.com/vincentfpgarcia/kNN-CUDA

Pointnet2.PyTorch: https://github.com/sshaoshuai/Pointnet2.PyTorch