Live Demonstration: Event-Driven Deep Neural Network Hardware System for Sensor Fusion

Abstract

We demonstrate an interactive digit recognition system using a spiking Deep Neural Network (DNN) FPGA-based system connected to two event-driven sensors: a Dynamic Vision Sensor (DVS) and a Dynamic Audio Sensor (DAS). Sensor fusion is demonstrated on a digit classification task using a DNN trained on the MNIST dataset supplemented by assignment of a unique pure ton e for each digit.