Crossing Nets: Combining GANs and VAEs with a Shared Latent Space for Hand Pose Estimation
OPEN ACCESS
Loading...
Author / Producer
Date
2017-10-09
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
State-of-the-art methods for 3D hand pose estimation from depth images require large amounts of annotated training data. We propose to model the statistical relationships of 3D hand poses and corresponding depth images using two deep generative models with a shared latent space. By design, our architecture allows for learning from unlabeled image data in a semi-supervised manner. Assuming a one-to-one mapping between a pose and a depth map, any given point in the shared latent space can be projected into both a hand pose and a corresponding depth map. Regressing the hand pose can then be done by learning a discriminator to estimate the posterior of the latent pose given some depth maps. To improve generalization and to better exploit unlabeled depth maps, we jointly train a generator and a discriminator. At each iteration, the generator is updated with the back-propagated gradient from the discriminator to synthesize realistic depth maps of the articulated hand, while the discriminator benefits from an augmented training set of synthesized and unlabeled samples. The proposed discriminator network architecture is highly efficient and runs at 90 FPS on the CPU with accuracies comparable or better than state-of-art on 3 publicly available benchmarks.
Comments: 10 pages, 5 figures, accepted in CVPR 2017
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:1702.03431 [cs.CV]
(or arXiv:1702.03431v2 [cs.CV] for this version)
Permanent link
Publication status
published
External links
Editor
Book title
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Journal / series
Volume
Pages / Article No.
Publisher
IEEE
Event
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
hand pose estimation from depth
Organisational unit
03514 - Van Gool, Luc (emeritus) / Van Gool, Luc (emeritus)
02652 - Institut für Bildverarbeitung / Computer Vision Laboratory