DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction
METADATA ONLY
Loading...
Author / Producer
Date
2020
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Neuromorphic event cameras are useful for dynamic vision problems under difficult lighting conditions. To enable studies of using event cameras in automobile driving applications, this paper reports a new end-to-end driving dataset called DDD20. The dataset was captured with a DAVIS camera that concurrently streams both dynamic vision sensor (DVS) brightness change events and active pixel sensor (APS) intensity frames. DDD20 is the longest event camera end-to-end driving dataset to date with 51h of DAVIS event+frame camera and vehicle human control data collected from 4000km of highway and urban driving under a variety of lighting conditions. Using DDD20, we report the first study of fusing brightness change events and intensity frame data using a deep learning approach to predict the instantaneous human steering wheel angle. Over all day and night conditions, the explained variance for human steering prediction from a Resnet-32 is significantly better from the fused DVS+APS frames (0.88) than using either DVS (0.67) or APS (0.77) data alone.
Permanent link
Publication status
published
Editor
Book title
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)
Journal / series
Volume
Pages / Article No.
9294515
Publisher
IEEE
Event
23rd International Conference on Intelligent Transportation Systems (ITSC 2020) (virtual)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
02533 - Institut für Neuroinformatik / Institute of Neuroinformatics
Notes
Due to the Coronavirus (COVID-19) the conference was conducted virtually.
Funding
Related publications and datasets
Is supplemented by: