DAVIS Driving Dataset 2020 (DDD20)

Abstract

Neuromorphic event cameras are useful for dynamic vision problems under difficult lighting conditions. To enable studies of using event cameras in automobile driving applications, this paper reports a new end-to-end driving dataset called DDD20. The dataset was captured with a DAVIS camera that concurrently streams both dynamic vision sensor (DVS) brightness change events and active pixel sensor (APS) intensity frames. DDD20 is the longest event camera end-toend driving dataset to date with 51h of DAVIS event+frame camera and vehicle human control data collected from 4000 km of highway and urban driving under a variety of lighting conditions. Using DDD20, we report the first study of fusing brightness change events and intensity frame data using a deep learning approach to predict the instantaneous human steering wheel angle. Over all day and night conditions, the explained variance for human steering prediction from a Resnet-32 is significantly better from the fused DVS+APS frames (0.88) than using either DVS (0.67) or APS (0.77) data alone.

DDD17 is the first public end-to-end training dataset of automotive driving using a DAVIS event+frame camera. It includes car data such as steering, throttle, GPS, etc. It can be used to evaluate the fusion of frame and event data for automobile driving applications.