Connecting the Dots: Combining Points and Lines for more Robust and Accurate Local Features
Open access
Author
Date
2023Type
- Doctoral Thesis
ETH Bibliography
yes
Altmetrics
Abstract
Local features are the building block of most multi-view applications such as localization and mapping, 3D reconstruction, and neural rendering, to obtain a highly accurate 3D geometry of a scene. Thus, local features are expected to be accurate, but also to be efficient and versatile to different tasks and conditions. This thesis explores some of the limitations of modern local features and offers solutions, with a particular focus on accuracy and robustness. It starts by reviewing a common issue of the widely used feature points: the trade-off between generalization and discrimination capabilities. A light-weight solution is proposed to select the most adapted invariance among several feature descriptors. Its hierarchical approach offers fine-grained robustness, while remaining compatible with both handcrafted and learned descriptors. In a second part, we tackle the challenges of the emerging line features and demonstrate their benefits in multiple tasks. We first propose a joint neural network to detect and describe line segments in images with high accuracy and robustness. Trained without ground truth labels, our line features are also equipped with a mechanism to handle partial occlusion. We then further improve the line detection with a hybrid approach combining deep learning for its robustness, and handcrafted strategies for their high accuracy. In a final part, we draw the conclusion that points and lines are complementary features and that they should be used in combination. We thus propose a joint deep matcher of points and lines, and show that leveraging the connectivity between all features is highly beneficial for a robust matching. We finally demonstrate how the combination of points and lines can be incorporated in a wide range of geometrical tasks and can boost their downstream performance. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000660085Publication status
publishedExternal links
Search print copy at ETH Library
Publisher
ETH ZurichSubject
Computer Vision; Local Features; Feature Matching; Visual LocalizationOrganisational unit
03766 - Pollefeys, Marc / Pollefeys, Marc
More
Show all metadata
ETH Bibliography
yes
Altmetrics