SeqNet: Learning Descriptors for Sequence-Based Hierarchical Place Recognition
[ArXiv+Supplementary] [IEEE Xplore RA-L 2021] [ICRA 2021 YouTube Video]
and
SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition
[ArXiv] [CVPR 2021 Workshop 3DVR]
Sequence-Based Hierarchical Visual Place Recognition.
News:
Jun 23: CVPR 2021 Workshop 3DVR paper, "SeqNetVLAD vs PointNetVLAD", now available on arXiv. Oxford dataset to be released soon.
Jun 02: SeqNet code release with the Nordland dataset.
Setup (One time)
Conda
conda create -n seqnet python=3.8 mamba -c conda-forge -y
conda activate seqnet
mamba install numpy pytorch=1.8.0 torchvision tqdm scikit-learn faiss tensorboardx h5py -c conda-forge -y
Download
Run bash download.sh to download single image NetVLAD descriptors (3.4 GB) for the Nordland-clean dataset [a] and corresponding model files (1.5 GB) [b].
Run
Train
To train sequential descriptors through SeqNet:
python main.py --mode train --pooling seqnet --dataset nordland-sw --seqL 10 --w 5 --outDims 4096 --expName "w5"
To (re-)train single descriptors through SeqNet:
python main.py --mode train --pooling seqnet --dataset nordland-sw --seqL 1 --w 1 --outDims 4096 --expName "w1"
Test
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-22-44_l10_w5/
The above will reproduce results for SeqNet (S5) as per Supp. Table III on Page 10.
To obtain other results from the same table, expand this.
# Raw Single (NetVLAD) Descriptor
python main.py --mode test --pooling single --dataset nordland-sf --seqL 1 --split test
# SeqNet (S1)
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 1 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/
# Raw + Smoothing
python main.py --mode test --pooling smooth --dataset nordland-sf --seqL 5 --split test
# Raw + Delta
python main.py --mode test --pooling delta --dataset nordland-sf --seqL 5 --split test
# Raw + SeqMatch
python main.py --mode test --pooling single+seqmatch --dataset nordland-sf --seqL 5 --split test
# SeqNet (S1) + SeqMatch
python main.py --mode test --pooling s1+seqmatch --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/
# HVPR (S5 to S1)
# Run S5 first and save its predictions by specifying `resultsPath`
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-22-44_l10_w5/ --resultsPath ./data/results/
# Now run S1 + SeqMatch using results from above (the timestamp of `predictionsFile` would be different in your case)
python main.py --mode test --pooling s1+seqmatch --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/ --predictionsFile ./data/results/Jun03_16-07-36_l5_0.npz
Acknowledgement
The code in this repository is based on Nanne/pytorch-NetVlad. Thanks to Tobias Fischer for his contributions to this code during the development of our project QVPR/Patch-NetVLAD.
Citation
@article{garg2021seqnet,
title={SeqNet: Learning Descriptors for Sequence-based Hierarchical Place Recognition},
author={Garg, Sourav and Milford, Michael},
journal={IEEE Robotics and Automation Letters},
volume={6},
number={3},
pages={4305-4312},
year={2021},
publisher={IEEE},
doi={10.1109/LRA.2021.3067633}
}
@misc{garg2021seqnetvlad,
title={SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition},
author={Garg, Sourav and Milford, Michael},
howpublished={CVPR 2021 Workshop on 3D Vision and Robotics (3DVR)},
month={Jun},
year={2021},
}
Other Related Projects
Patch-NetVLAD (2021); Delta Descriptors (2020); CoarseHash (2020); seq2single (2019); LoST (2018)
[a] This is the clean version of the dataset that excludes images from the tunnels and red lights, exact image names can be obtained from here.
[b] These will automatically save to ./data/, you can modify this path in download.sh and get_datasets.py to specify your workdir.
1 Feb 11, 2022
1.4k Dec 29, 2022
1 Feb 08, 2022
399 Jan 08, 2023
632 Dec 26, 2022
1.1k Dec 16, 2022
56 Jan 02, 2023
4 Mar 15, 2022
53 Dec 13, 2022
37 Dec 15, 2022
38 Jan 01, 2023
52 Dec 04, 2022
198 Dec 27, 2022
215 Dec 30, 2022
2.7k Jan 07, 2023
168 Jan 05, 2023
22 Dec 12, 2022
3 Jul 10, 2022
423 Dec 07, 2022
37 Dec 27, 2022