PyTorch source code for the Variational Pose Regression method proposed in the ICRA 2023 paper A Probabilistic Framework for Visual Localization in Ambiguous Scenes.
All python dependencies of the project are listed in the requirement.txt file, installed by pip install -r requirements.txt. In addition to these, and optionally
- Deep Bingham Networks package is required for computing sample likelihoods,
- Instant Neural Graphics Primitives package is required for modelling scenes and view synthesis from sampled poses, and
- Weights & Biases for logging of results.
If you do not wish to install the latter packages, do not run/comment out the relevant sections that use those functionalities.
Clone the repository and install it, for example, with
pip install .The project codebase assumes the following file structure,
+── ~/data
│ +── AmbiguousReloc
│ +── blue_chairs
│ +── meeting_table
│ +── seminar
│ +── staircase
│ +── staircase_ext
│ +── CambridgeLandmarks
│ +── KingsCollege
│ +── OldHospital
│ +── ShopFacade
│ +── StMarysChurch
│ +── Street
│ +── SevenScenes
│ +── chess
│ +── fire
│ +── heads
│ +── office
│ +── pumpkin
│ +── redkitchen
│ +── stairs
│ +── Rig
│ +── Ceiling
+── ~/code
│ +── vapor (clone root)
│ +── vapor
│ +── scripts
│ +── runs
where each dataset follows its own structure for images and ground truth pose labels. Refer to vapor/data.py for the exact structures.
Our recorded data for the ceiling scene can be downloaded here. Ground-truth poses are saved in the poses.txt file inside each sequence directory, where each line contains the camera pose (camera-to-world point transformation) for one image sample with the following file format:
seq_no image_no tx ty tz qw qx qy qz
The project contains various tools for evaluation and analysis of results, including many tools that were not included in the paper. A minimal guide to train and evaluate the pipeline is provided below. However, feel free to explore the available tools.
You can train the pipeline by the training script
python vapor/scripts/train_pipeline.py --dataset AmbiguousReloc --sequence blue_chairs
and provide the appropriate settings by flags. The results will be saved at vapor/runs/RUNNAME.
You can evaluate a pretrained model by the evaluation script
python vapor/scripts/evaluate_pipeline.py RUNNAME
You can then visualize the samples predicted by the pretrained model by the visualization script
python vapor/scripts/visualize_samples.py RUNNAME
The results will be saved at vapor/runs/RUNNAME/plots/.
If you find this library useful in your research, consider citing our publication:
@inproceedings{zangeneh2023vapor,
title={A Probabilistic Framework for Visual Localization in Ambiguous Scenes},
author={Zangeneh, Fereidoon and Bruns, Leonard and Dekel, Amit and Pieropan, Alessandro and Jensfelt, Patric},
booktitle={Proceedings of the IEEE International Conference on Robotics and Automation},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent