Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer

Open access
Date
2022-03Type
- Journal Article
Citations
Cited 130 times in
Web of Science
Cited 71 times in
Scopus
ETH Bibliography
yes
Altmetrics
Abstract
The success of monocular depth estimation relies on large and diverse training sets. Due to the challenges associated with acquiring dense ground-truth depth across different environments at scale, a number of datasets with distinct characteristics and biases have emerged. We develop tools that enable mixing multiple datasets during training, even if their annotations are incompatible. In particular, we propose a robust training objective that is invariant to changes in depth range and scale, advocate the use of principled multi-objective learning to combine data from different sources, and highlight the importance of pretraining encoders on auxiliary tasks. Armed with these tools, we experiment with six diverse training datasets, including a new, massive data source: 3D films. To demonstrate the generalization power of our approach we use zero-shot cross-dataset transfer, i.e. we evaluate on datasets that were not seen during training. The experiments confirm that mixing data from complementary sources greatly improves monocular depth estimation. Our approach clearly outperforms competing methods across diverse datasets, setting a new state of the art for monocular depth estimation. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000462024Publication status
publishedJournal / series
IEEE Transactions on Pattern Analysis and Machine IntelligenceVolume
Pages / Article No.
Publisher
IEEESubject
Monocular depth estimation; Single-image depth prediction; Zero-shot cross-dataset transfer; Multi-dataset trainingOrganisational unit
03886 - Schindler, Konrad / Schindler, Konrad
More
Show all metadata
Citations
Cited 130 times in
Web of Science
Cited 71 times in
Scopus
ETH Bibliography
yes
Altmetrics