Denys Rozumnyi

Last Name

Rozumnyi

First Name

Denys

ORCID

0000-0001-9874-1349

Show all metadata

Search Results

Publications 1 - 10 of 14

Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos
Rozumnyi, Denys; Oswald, Martin R.; Ferrari, Vittorio; et al. (2022)
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. To this end, we model the blurred appearance of a fast moving object in a generative fashion by parametrizing its 3D position, rotation, velocity, acceleration, bounces, shape, and texture over the duration of a predefined time window spanning multiple frames. Using differentiable rendering, we are able to estimate all parameters by minimizing the pixel-wise reprojection error to the input video via backpropagating through a rendering pipeline that accounts for motion blur by averaging the graphics output over short time intervals. For that purpose, we also estimate the camera exposure gap time within the same optimization. To account for abrupt motion changes like bounces, we model the motion trajectory as a piece-wise polynomial, and we are able to estimate the specific time of the bounce at sub-frame accuracy. Experiments on established benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.
Human from Blur: Human Pose Tracking from Blurry Images
Zhao, Yiming; Rozumnyi, Denys; Song, Jie; et al. (2023)
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
We propose a method to estimate 3D human poses from substantially blurred images. The key idea is to tackle the inverse problem of image deblurring by modeling the forward problem with a 3D human model, a texture map, and a sequence of poses to describe human motion. The blurring process is then modeled by a temporal image aggregation step. Using a differentiable renderer, we can solve the inverse problem by backpropagating the pixel-wise reprojection error to recover the best human motion representation that explains a single or multiple input images. Since the image reconstruction loss alone is insufficient, we present additional regularization terms. To the best of our knowledge, we present the first method to tackle this problem. Our method consistently outperforms other methods on significantly blurry inputs since they lack one or multiple key functionalities that our method unifies, i.e. image deblurring with sub-frame accuracy and explicit 3D modeling of non-rigid human motion.
FMODetect: Robust Detection of Fast Moving Objects
Rozumnyi, Denys; Matas, Jiri; Šroubek, Filip; et al. (2021)
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
We propose the first learning-based approach for fast moving objects detection. Such objects are highly blurred and move over large distances within one video frame. Fast moving objects are associated with a deblurring and matting problem, also called deblatting. We show that the separation of deblatting into consecutive matting and deblurring allows achieving real-time performance, i.e. an order of magnitude speed-up, and thus enabling new classes of application. The proposed method detects fast moving objects as a truncated distance function to the trajectory by learning from synthetic data. For the sharp appearance estimation and accurate trajectory estimation, we propose a matting and fitting network that estimates the blurred appearance without background, followed by an energy minimization based deblurring. The state-of-the-art methods are outperformed in terms of recall, precision, trajectory estimation, and sharp appearance reconstruction. Compared to other methods, such as deblatting, the inference is of several orders of magnitude faster and allows applications such as real-time fast moving object detection and retrieval in large video collections.
Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects
Rozumnyi, Denys; Oswald, Martin R.; Ferrari, Vittorio; et al. (2021)
Advances in Neural Information Processing Systems 34
We address the novel task of jointly reconstructing the 3D shape, texture, and motion of an object from a single motion-blurred image. While previous approaches address the deblurring problem only in the 2D image domain, our proposed rigorous modeling of all object properties in the 3D domain enables the correct description of arbitrary object motion. This leads to significantly better image decomposition and sharper deblurring results. We model the observed appearance of a motion-blurred object as a combination of the background and a 3D object with constant translation and rotation. Our method minimizes a loss on reconstructing the input image via differentiable rendering with suitable regularizers. This enables estimating the textured 3D mesh of the blurred object with high fidelity. Our method substantially outperforms competing approaches on several benchmarks for fast moving objects deblurring. Qualitative results show that the reconstructed 3D mesh generates high-quality temporal super-resolution and novel views of the deblurred object.
Retrieval Robust to Object Motion Blur
Zou, Rong; Pollefeys, Marc; Rozumnyi, Denys (2025)
Lecture Notes in Computer Science ~ Computer Vision – ECCV 2024
Moving objects are frequently seen in daily life and usually appear blurred in images due to their motion. While general object retrieval is a widely explored area in computer vision, it primarily focuses on sharp and static objects, and retrieval of motion-blurred objects in large image collections remains unexplored. We propose a method for object retrieval in images that are affected by motion blur. The proposed method learns a robust representation capable of matching blurred objects to their deblurred versions and vice versa. To evaluate our approach, we present the first large-scale datasets for blurred object retrieval, featuring images with objects exhibiting varying degrees of blur in various poses and scales. We conducted extensive experiments, showing that our method outperforms state-of-the-art retrieval methods on the new blur-retrieval datasets, which validates the effectiveness of the proposed approach. Code, data, and model are available at https://github.com/Rong-Zou/Retrieval-Robust-to-Object-Motion-Blur.
Learned Semantic Multi-Sensor Depth Map Fusion
Rozumnyi, Denys; Cherabier, Ian; Pollefeys, Marc; et al. (2020)
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Reconstructing Motion-Blurred Objects
Rozumnyi, Denys (2024)
Objects moving at high speed appear significantly blurred when captured by cameras. This blurry appearance becomes particularly ambiguous when the object has a complex shape or texture. In such cases, classical methods, and even humans, often fail to recover the object’s appearance and motion. In this thesis, we propose a comprehensive range of methods to address the challenges associated with motion-blurred objects. First, we address the retrieval and detection of these objects. Next, we present methods to track them, whether they are blurred or not. We then develop techniques for deblurring and temporal super-resolution, producing the object’s appearance and position in a series of sub-frames, as if captured by a high-speed camera. As the culmination of this research, we introduce a novel task and approach for jointly reconstructing the 3D shape, texture, and motion of an object from a single motion-blurred image. Our rigorous modeling of all object properties in the 3D domain enables the correct description of arbitrary object motion. This is achieved by minimizing a loss function that reconstructs the input image via differentiable rendering with appropriate regularizers. This enables the estimation of a high-fidelity textured 3D mesh of the blurred object. We address cases involving a single input frame, multiple frames, and even instances where the object is a human body. In the latter case, the key idea is to solve the inverse problem of image deblurring by modeling the forward problem using a 3D human model, a texture map, and a sequence of poses to describe human motion. Experiments on newly established benchmark datasets demonstrate that the proposed methods significantly outperform competing approaches for motion-blurred object retrieval, detection, tracking, deblurring, and 3D reconstruction. A wide range of applications and use cases illustrate the impact of the methods proposed in this thesis.
DeFMO: Deblurring and Shape Recovery of Fast Moving Objects
Rozumnyi, Denys; Oswald, Martin R.; Ferrari, Vittorio; et al. (2021)
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Objects moving at high speed appear significantly blurred when captured with cameras. The blurry appearance is especially ambiguous when the object has complex shape or texture. In such cases, classical methods, or even humans, are unable to recover the object’s appearance and motion. We propose a method that, given a single image with its estimated background, outputs the object’s appearance and position in a series of sub-frames as if captured by a high-speed camera (i.e. temporal super-resolution). The proposed generative model embeds an image of the blurred object into a latent space representation, disentangles the background, and renders the sharp appearance. Inspired by the image formation model, we design novel self-supervised loss function terms that boost performance and show good generalization capabilities. The proposed DeFMO method is trained on a complex synthetic dataset, yet it performs well on real-world data from several datasets. DeFMO outperforms the state of the art and generates high-quality temporal super-resolution frames.
Single-Image Deblurring, Trajectory and Shape Recovery of Fast Moving Objects with Denoising Diffusion Probabilistic Models
Spetlik, Radim; Rozumnyi, Denys; Matas, Jiří (2024)
2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Blurry appearance of fast moving objects in video frames was successfully used to reconstruct the object appearance and motion in both 2D and 3D domains. The proposed method addresses the novel, severely ill-posed, task of single-image fast moving object deblurring, shape, and trajectory recovery - previous approaches require at least three consecutive video frames. Given a single image, the method outputs the object 2D appearance and position in a series of sub-frames as if captured by a high-speed camera (i.e. temporal super-resolution). The proposed SI-DDPM-FMO method is trained end-to-end on a synthetic dataset with various moving objects, yet it generalizes well to real-world data from several publicly available datasets. SI-DDPM-FMO performs similarly to or better than recent multi-frame methods and a carefully designed baseline method.
Sub-Frame Appearance and 6D Pose Estimation of Fast Moving Objects
Rozumnyi, Denys; Kotera, Jan; Šroubek, Filip; et al. (2020)
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We propose a novel method that tracks fast moving objects, mainly non-uniform spherical, in full 6 degrees of freedom, estimating simultaneously their 3D motion trajectory, 3D pose and object appearance changes with a time step that is a fraction of the video frame exposure time. The sub-frame object localization and appearance estimation allows realistic temporal super-resolution and precise shape estimation. The method, called TbD-3D (Tracking by Deblatting in 3D) relies on a novel reconstruction algorithm which solves a piece-wise deblurring and matting problem. The 3D rotation is estimated by minimizing the reprojection error. As a second contribution, we present a new challenging dataset with fast moving objects that change their appearance and distance to the camera. High-speed camera recordings with zero lag between frame exposures were used to generate videos with different frame rates annotated with ground-truth trajectory and pose.

Publications 1 - 10 of 14

Denys Rozumnyi

Last Name

First Name

ORCID

Organisational unit

Filters

Search Results