arXiv pre-print View it on Github

FaVoR: Features via Voxel Rendering for Camera Relocalization

Vincenzo Polizzi¹, Marco Cannici², Davide Scaramuzza², Jonathan Kelly¹

¹University of Toronto, ²University of Zurich

Abstract

Cambridge Landmarks Visualization

+

+

+

+

+

×

Great Court

The video below shows the camera pose relocalization computed using FaVoR. The purple frame indicates the starting camera position provided by the first DenseVLAD result, while the blue frame represents the ground truth camera pose of the query image. The estimated camera pose is shown in black, connected to the initial pose by a green line.

Move the slidebar to change camera pose.

×

King's College

The video below shows the camera pose relocalization computed using FaVoR. The purple frame indicates the starting camera position provided by the first DenseVLAD result, while the blue frame represents the ground truth camera pose of the query image. The estimated camera pose is shown in black, connected to the initial pose by a green line.

Move the slidebar to change camera pose.

×

Old Hospital

The video below shows the camera pose relocalization computed using FaVoR. The purple frame indicates the starting camera position provided by the first DenseVLAD result, while the blue frame represents the ground truth camera pose of the query image. The estimated camera pose is shown in black, connected to the initial pose by a green line.

Move the slidebar to change camera pose.

×

Shop Facade

The video below shows the camera pose relocalization computed using FaVoR. The purple frame indicates the starting camera position provided by the first DenseVLAD result, while the blue frame represents the ground truth camera pose of the query image. The estimated camera pose is shown in black, connected to the initial pose by a green line.

Move the slidebar to change camera pose.

×

St. Mary's Church

The video below shows the camera pose relocalization computed using FaVoR. The purple frame indicates the starting camera position provided by the first DenseVLAD result, while the blue frame represents the ground truth camera pose of the query image. The estimated camera pose is shown in black, connected to the initial pose by a green line.

Move the slidebar to change camera pose.

FaVoR vs. Standard Features Matcher

7-Scenes Chess, features invariance

In the video below, we extract Alike-l features from a fixed target image and match these features with those extracted from a query image using standard feature matching. On the right side, we display the matches from three iterations of the FaVoR method, where FaVoR is queried using the fixed target's pose. It is noticeable that the number of matches significantly increases in the third iteration of FaVoR compared to the standard matching approach. The text at the bottom left of the image shows the distance (in meters and degrees) between the target and query images, as well as the number of matches for both methods. The text turns red when the number of standard feature matches are more than the FaVoR matches.

Rendering Capabilities

To evaluate the view invariance of feature descriptors, we extract dense descriptor maps from images taken at different angles. Using Alike-l, we compute similarity scores between features from a target image and the dense maps. The same process is applied to FaVoR, using the ground truth pose for rendering. The figure below shows the median similarity scores of the top thirty matches for both Alike-l and FaVoR across different query angles. FaVoR maintains nearly constant scores, indicating good descriptor fidelity from unseen views. In contrast, Alike-l shows a noticeable drop in similarity beyond ±30 degrees, highlighting the advantage of FaVoR in maintaining descriptor consistency across viewpoints.

In blue is the smoothed median score for FaVoR_Alike-l obtained by convolving the descriptors rendered at different view angles with the corresponding dense descriptors map of each query image. In orange is the smoothed median score of Alike-l features extracted from the starting image (at angle 0 deg) convolved with the subsequent images in the test sequence.

Results

7-Scenes Dataset

6-DoF median localization errors on the 7-Scenes dataset. Comparison of visual localization methods. The overall top three results are shown in bold, underline, and double-underline.

Cambridge Landmarks Dataset

6-DoF median localization errors on the Cambridge Landmarks dataset. Comparison of visual localization methods. The overall top three results are shown in bold, underline, and double-underline.

Models Downlaod

Coming soon!

7-Scenes Dataset

Cambridge Landmarks Dataset

Cite this work

@misc{polizzi2024arXiv,
    title={FaVoR: Features via Voxel Rendering for Camera Relocalization}, 
    author={Vincenzo Polizzi and Marco Cannici and Davide Scaramuzza and Jonathan Kelly},
    year={2024},
    eprint={2409.07571},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2409.07571}, 
}