The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
OPEN ACCESS
Loading...
Author / Producer
Date
2021
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Deep learning has enabled impressive progress in the accuracy of semantic segmentation. Yet, the ability to estimate uncertainty and detect failure is key for safety-critical applications like autonomous driving. Existing uncertainty estimates have mostly been evaluated on simple tasks, and it is unclear whether these methods generalize to more complex scenarios. We present Fishyscapes, the first public benchmark for anomaly detection in a real-world task of semantic segmentation for urban driving. It evaluates pixel-wise uncertainty estimates towards the detection of anomalous objects. We adapt state-of-the-art methods to recent semantic segmentation models and compare uncertainty estimation approaches based on softmax confidence, Bayesian learning, density estimation, image resynthesis, as well as supervised anomaly detection methods. Our results show that anomaly detection is far from solved even for ordinary situations, while our benchmark allows measuring advancements beyond the state-of-the-art. Results, data and submission information can be found at https://fishyscapes.com/.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
129 (11)
Pages / Article No.
3119 - 3135
Publisher
Springer
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Semantic segmentation; Anomaly detection; Uncertainty estimation; Out-of-distribution detection; Autonomous driving
Organisational unit
03737 - Siegwart, Roland Y. / Siegwart, Roland Y.
Notes
Funding
Related publications and datasets
Is part of: