Reconstructing Motion-Blurred Objects
OPEN ACCESS
Loading...
Author / Producer
Date
2024
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Objects moving at high speed appear significantly blurred when captured by cameras. This blurry appearance becomes particularly ambiguous when the object has a complex shape or texture. In such cases, classical methods, and even humans, often fail to recover the object’s appearance and motion. In this thesis, we propose a comprehensive range of methods to address the challenges associated with motion-blurred objects.
First, we address the retrieval and detection of these objects. Next, we present methods to track them, whether they are blurred or not. We then develop techniques for deblurring and temporal super-resolution, producing the object’s appearance and position in a series of sub-frames, as if captured by a high-speed camera.
As the culmination of this research, we introduce a novel task and approach for jointly reconstructing the 3D shape, texture, and motion of an object from a single motion-blurred image. Our rigorous modeling of all object properties in the 3D domain enables the correct description of arbitrary object motion. This is achieved by minimizing a loss function that reconstructs the input image via differentiable rendering with appropriate regularizers. This enables the estimation of a high-fidelity textured 3D mesh of the blurred object.
We address cases involving a single input frame, multiple frames, and even instances where the object is a human body. In the latter case, the key idea is to solve the inverse problem of image deblurring by modeling the forward problem using a 3D human model, a texture map, and a sequence of poses to describe human motion.
Experiments on newly established benchmark datasets demonstrate that the proposed methods significantly outperform competing approaches for motion-blurred object retrieval, detection, tracking, deblurring, and 3D reconstruction. A wide range of applications and use cases illustrate the impact of the methods proposed in this thesis.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Pollefeys, Marc
Examiner : Oswald, Martin R.
Examiner : Ferrari, Vittorio
Examiner : Vedaldi, Andrea
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
computer vision; machine learning; 3d reconstruction; deblurring; motion blur; object tracking
Organisational unit
03766 - Pollefeys, Marc / Pollefeys, Marc