EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices
METADATA ONLY
Loading...
Author / Producer
Date
2022
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Understanding social interactions from egocentric views is crucial for many applications, ranging from assistive robotics to AR/VR. Key to reasoning about interactions is to understand the body pose and motion of the interaction partner from the egocentric view. However, research in this area is severely hindered by the lack of datasets. Existing datasets are limited in terms of either size, capture/annotation modalities, ground-truth quality, or interaction diversity. We fill this gap by proposing EgoBody, a novel large-scale dataset for human pose, shape and motion estimation from egocentric views, during interactions in complex 3D scenes. We employ Microsoft HoloLens2 headsets to record rich egocentric data streams (including RGB, depth, eye gaze, head and hand tracking). To obtain accurate 3D ground truth, we calibrate the headset with a multi-Kinect rig and fit expressive SMPL-X body meshes to multi-view RGB-D frames, reconstructing 3D human shapes and poses relative to the scene, over time. We collect 125 sequences, spanning diverse interaction scenarios, and propose the first benchmark for 3D full-body pose and shape estimation of the interaction partner from egocentric views. We extensively evaluate state-of-the-art methods, highlight their limitations in the egocentric scenario, and address such limitations leveraging our high-quality annotations. Data and code are available at https://sanweiliti.github.io/egobody/egobody.html.
Permanent link
Publication status
published
External links
Book title
Computer Vision – ECCV 2022
Journal / series
Volume
13666
Pages / Article No.
180 - 200
Publisher
Springer
Event
European Conference on Computer Vision (ECCV 2022)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Pose estimation; Egocentric view; Motion capture; Dataset
Organisational unit
03766 - Pollefeys, Marc / Pollefeys, Marc
09686 - Tang, Siyu / Tang, Siyu
Notes
Funding
204840 - Learning to Create Realistic Human Avatars (SNF)