Hi4D: 4D Instance Segmentation of Close Human Interaction
METADATA ONLY
Loading...
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact. Robustly disentangling several in-contact subjects is a challenging task due to occlusions and complex shapes. Hence, existing multi-view systems typically fuse 3D surfaces of close subjects into a single, connected mesh. To address this issue we leverage i) individually fitted neural implicit avatars; ii) an alternating optimization scheme that refines pose and surface through periods of close proximity; and iii) thus segment the fused raw scans into individual instances. From these instances we compile Hi4D dataset of 4D textured scans of 20 subject pairs, 100 sequences, and a total of more than 11 K frames. Hi4D contains rich interaction-centric annotations in 2D and 3D alongside accurately registered parametric body models. We define varied human pose and shape estimation tasks on this dataset and provide results from state-of-the-art methods on these benchmarks.
Permanent link
Publication status
published
External links
Editor
Book title
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Journal / series
Volume
Pages / Article No.
17016 - 17027
Publisher
IEEE
Event
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03979 - Hilliges, Otmar (ehemalig) / Hilliges, Otmar (former)