Ternarized TCN for μJ/Inference Gesture Recognition from DVS Event Frames
OPEN ACCESS
Loading...
Author / Producer
Date
2022
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Abstract
Dynamic Vision Sensors (DVS) offer the opportunity to scale the energy consumption in image acquisition proportionally to the activity in the captured scene by only transmitting data when the captured image changes. Their potential for energy-proportional sensing makes them highly attractive for severely energy-constrained sensing nodes at the edge. Most approaches to the processing of DVS data employ Spiking Neural Networks to classify the input from the sensor. In this paper, we propose an alternative, event frame-based approach to the classification of DVS video data. We assemble ternary video frames from the event stream and process them with a fully ternarized Temporal Convolutional Network which can be mapped to CUTIE, a highly energy-efficient Ternary Neural Network accelerator. The network mapped to the accelerator achieves a classification accuracy of 94.5%, matching the state of the art for embedded implementations. We implement the processing pipeline in a modern 22 nm FDX technology and perform post-synthesis power simulation of the network running on the system, achieving an inference energy of 1.7 μJ which is 647x lower than previously reported results based on Spiking Neural Networks.
Permanent link
Publication status
published
Editor
Book title
2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Journal / series
Volume
Pages / Article No.
736 - 741
Publisher
IEEE
Event
25th Design, Automation and Test in Europe Conference (DATE 2022)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Dynamic vision sensor (DVS); Computer Vision; Neural network
Organisational unit
03996 - Benini, Luca / Benini, Luca
Notes
Conference lecture held on 22 March 2022.
Funding
877056 - A Cognitive Fractal and Secure EDGE based on an unique Open-Safe-Reliable-Low Power Hardware Platform Node (EC)
180625 - Heterogeneous Computing Systems with Customized Accelerators (SNF)
180625 - Heterogeneous Computing Systems with Customized Accelerators (SNF)