Search
Results
-
SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding
(2023)Advances in Neural Information Processing Systems 36Semantic 2D maps are commonly used by humans and machines for navigation purposes, whether it's walking or driving. However, these maps have limitations: they lack detail, often contain inaccuracies, and are difficult to create and maintain, especially in an automated fashion. Can we use raw imagery to automatically create better maps that can be easily interpreted by both humans and machines? We introduce SNAP, a deep network that learns ...Conference Paper -
LightGlue: Local Feature Matching at Light Speed
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)We introduce LightGlue, a deep neural network that learns to match local features across images. We revisit multiple design decisions of SuperGlue, the state of the art in sparse matching, and derive simple but effective improvements. Cumulatively, they make LightGlue more efficient -- in terms of both memory and computation, more accurate, and much easier to train. One key property is that LightGlue is adaptive to the difficulty of the ...Conference Paper -
R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)Dense 3D reconstruction and ego-motion estimation are key challenges in autonomous driving and robotics. Compared to the complex, multi-modal systems deployed today, multi-camera systems provide a simpler, low-cost alternative. However, camera-based 3D reconstruction of complex dynamic scenes has proven extremely difficult, as existing solutions often produce incomplete or incoherent results. We propose R3D3, a multi-camera system for ...Conference Paper -
RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registration
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)Although point cloud registration has achieved remarkable advances in object-level and indoor scenes, large-scale registration methods are rarely explored. Challenges mainly arise from the huge point number, complex distribution, and outliers of outdoor LiDAR scans. In addition, most existing registration works generally adopt a two-stage paradigm: They first find correspondences by extracting discriminative local features and then leverage ...Conference Paper -
RLSAC: Reinforcement Learning enhanced Sample Consensus for End-to-End Robust Estimation
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)Robust estimation is a crucial and still challenging task, which involves estimating model parameters in noisy environments. Although conventional sampling consensus-based algorithms sample several times to achieve robustness, these algorithms cannot use data features and historical information effectively. In this paper, we propose RLSAC, a novel Reinforcement Learning enhanced SAmple Consensus framework for end-to-end robust estimation. ...Conference Paper -
Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)We tackle the problem of estimating a Manhattan frame, i.e. three orthogonal vanishing points, and the unknown focal length of the camera, leveraging a prior vertical direction. The direction can come from an Inertial Measurement Unit that is a standard component of recent consumer devices, e.g., smartphones. We provide an exhaustive analysis of minimal line configurations and derive two new 2-line solvers, one of which does not suffer ...Conference Paper -
SGAligner: 3D Scene Alignment with Scene Graphs
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)Building 3D scene graphs has recently emerged as a topic in scene representation for several embodied AI applications to represent the world in a structured and rich manner. With their increased use in solving downstream tasks (e.g., navigation and room rearrangement), can we leverage and recycle them for creating 3D maps of environments, a pivotal step in agent operation? We focus on the fundamental problem of aligning pairs of 3D scene ...Conference Paper -
Privacy Preserving Localization via Coordinate Permutations
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)Recent methods on privacy-preserving image-based localization use a random line parameterization to protect the privacy of query images and database maps. The lifting of points to lines effectively drops one of the two geometric constraints traditionally used with point-to-point correspondences in structure-based localization. This leads to a significant loss of accuracy for the privacy-preserving methods. In this paper, we overcome this ...Conference Paper -
Guiding Local Feature Matching with Surface Curvature
(2023)2023 IEEE/CVF International Conference on Computer Vision (ICCV)We propose a new method, named curvature similarity extractor (CSE), for improving local feature matching across images. CSE calculates the curvature of the local 3D surface patch for each detected feature point in a viewpoint-invariant manner via fitting quadrics to predicted monocular depth maps. This curvature is then leveraged as an additional signal in feature matching with off-the-shelf matchers like SuperGlue and LoFTR. Additionally, ...Conference Paper -
The Drunkard’s Odometry: Estimating Camera Motion in Deforming Scenes
(2023)Advances in Neural Information Processing Systems 36Conference Paper