Learning Motion Priors for 4D Human Body Capture in 3D Scenes
Author / Producer
Date
2021
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
Data
Rights / License
Abstract
Recovering high-quality 3D human motion in complex scenes from monocular videos is important for many applications, ranging from AR/VR to robotics. However, capturing realistic human-scene interactions, while dealing with occlusions and partial views, is challenging; current approaches are still far from achieving compelling results. We address this problem by proposing LEMO: LEarning human MOtion priors for 4D human body capture. By leveraging the large-scale motion capture dataset AMASS, we introduce a novel motion smoothness prior, which strongly reduces the jitters exhibited by poses recovered over a sequence. Furthermore, to handle contacts and occlusions occurring frequently in body-scene interactions, we design a contact friction term and a contact-aware motion infiller obtained via per-instance self-supervised training. To prove the effectiveness of the proposed motion priors, we combine them into a novel pipeline for 4D human body capture in 3D scenes. With our pipeline, we demonstrate high-quality 4D human body capture, reconstructing smooth motions and physically plausible body-scene interactions. The code and data are available at https://sanweiliti.github.io/LEMO/LEMO.html.
Permanent link
Publication status
published
Editor
Book title
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Journal / series
Volume
Pages / Article No.
11323 - 11333
Publisher
IEEE
Event
18th International Conference on Computer Vision (ICCV 2021)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
deep learning; computer vision; 3D Vision
Organisational unit
09686 - Tang, Siyu / Tang, Siyu
03766 - Pollefeys, Marc / Pollefeys, Marc