Search
Results
-
VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction
(2023)2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)The success of the Neural Radiance Fields (NeRF) in novel view synthesis has inspired researchers to propose neural implicit scene reconstruction. However, most existing neural implicit reconstruction methods optimize perscene parameters and therefore lack generalizability to new scenes. We introduce VolRecon, a novel generalizable implicit reconstruction method with Signed Ray Distance Function (SRDF). To reconstruct the scene with fine ...Conference Paper -
ARrow: A Real-Time AR Rowing Coach
(2023)EuroVis 2023 - Short PapersRowing requires physical strength and endurance in athletes as well as a precise rowing technique. The ideal rowing stroke is based on biomechanical principles and typically takes years to master. Except for time-consuming video analysis after practice, coaches currently have no means to quantitatively analyze a rower's stroke sequence and body movement. We propose ARrow, an AR application for coaches and athletes that provides real-time ...Conference Paper -
Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors
(2023)2023 IEEE International Conference on Robotics and Automation (ICRA)A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks, such as image matching, image retrieval, and visual localization. State-of-the-art descriptors, from hand-crafted descriptors such as SIFT to learned ones such as HardNet, are usually high-dimensional; 128 dimensions or even more. The higher the dimensionality, the larger the memory consumption and computational ...Conference Paper -
Learning-based Relational Object Matching Across Views
(2023)2023 IEEE International Conference on Robotics and Automation (ICRA)Intelligent robots require object-level scene understanding to reason about possible tasks and interactions with the environment. Moreover, many perception tasks such as scene reconstruction, image retrieval, or place recognition can benefit from reasoning on the level of objects. While keypoint-based matching can yield strong results for finding correspondences for images with small to medium view point changes, for large view point ...Conference Paper -
SGAligner: 3D Scene Alignment with Scene Graphs
(2023)2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023)Building 3D scene graphs has recently emerged as a topic in scene representation for several embodied AI applications to represent the world in a structured and rich manner. With their increased use in solving downstream tasks (e.g., navigation and room rearrangement), can we leverage and recycle them for creating 3D maps of environments, a pivotal step in agent operation? We focus on the fundamental problem of aligning pairs of 3D scene ...Conference Paper -
Guiding Local Feature Matching with Surface Curvature
(2023)2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023)We propose a new method, called curvature similarity extractor (CSE), for improving local feature matching across images. CSE calculates the curvature of the local 3D surface patch for each detected feature point in a viewpoint-invariant manner via fitting quadrics to predicted monocular depth maps. This curvature is then leveraged as an additional signal in feature matching with off-the-shelf matchers like SuperGlue and LoFTR. Additionally, ...Conference Paper -
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
(2023)2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023)Building an interactive AI assistant that can perceive, reason, and collaborate with humans in the real world has been a long-standing pursuit in the AI community. This work is part of a broader research effort to develop intelligent agents that can interactively guide humans through performing tasks in the physical world. As a first step in this direction, we introduce HoloAssist, a large-scale egocentric human interaction dataset, where ...Conference Paper -
Privacy Preserving Localization via Coordinate Permutations
(2023)2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023)Recent methods on privacy-preserving image-based localization use a random line parameterization to protect the privacy of query images and database maps. The lifting of points to lines effectively drops one of the two geometric constraints traditionally used with point-to-point correspondences in structure-based localization. This leads to a significant loss of accuracy for the privacy-preserving methods. In this paper, we overcome this ...Conference Paper -
LightGlue: Local Feature Matching at Light Speed
(2023)2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023)We introduce LightGlue, a deep neural network that learns to match local features across images. We revisit multiple design decisions of SuperGlue, the state of the art in sparse matching, and derive simple but effective improvements. Cumulatively, they make LightGlue more efficient - in terms of both memory and computation, more accurate, and much easier to train. One key property is that LightGlue is adaptive to the difficulty of the ...Conference Paper -
Capturing and Animation of Body and Clothing from Monocular Video
(2022)SA ’22: SIGGRAPH Asia 2022 Conference PapersWhile recent work has shown progress on extracting clothed 3D human avatars from a single image, video, or a set of 3D scans, several limitations remain. Most methods use a holistic representation to jointly model the body and clothing, which means that the clothing and body cannot be separated for applications like virtual try-on. Other methods separately model the body and clothing, but they require training from a large set of 3D clothed ...Conference Paper