FrameFire: Enabling Efficient Spiking Neural Network Inference for Video Segmentation
METADATA ONLY
Loading...
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Fast video recognition is essential for real-time scenarios, e.g., autonomous driving. However, applying existing Deep Neural Networks (DNNs) to individual high-resolution images is expensive due to large model sizes. Spiking Neural Networks (SNNs) are developed as a promising alternative to DNNs due to their more realistic brain-inspired computing models. SNNs have sparse neuron firing over time, i.e., spatio-temporal sparsity; thus they are useful to enable energy-efficient computation. However, exploiting the spatio-temporal sparsity of SNNs in hardware leads to unpredictable and unbalanced workloads, degrading energy efficiency. In this work, we, therefore, propose an SNN accelerator called FrameFire for efficient video processing. We introduce a Keyframe-dominated Workload Balance Schedule (KWBS) method. It accelerates the image recognition network with sparse keyframes, then records and analyzes the current workload distribution on hardware to facilitate scheduling workloads in subsequent frames. FrameFire is implemented on a Xilinx XC7Z035 FPGA and verified by video segmentation tasks. The results show that the throughput is improved by 1.7× with the KWBS method. FrameFire achieved 1.04 KFPS throughput and 1.15 mJ/frame recognition energy.
Permanent link
Publication status
published
Editor
Book title
2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS)
Journal / series
Volume
Pages / Article No.
10168660
Publisher
IEEE
Event
5th International Conference on Artificial Intelligence Circuits and Systems (AICAS 2023)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Workload balance; SNNs; VLSI
Organisational unit
02533 - Institut für Neuroinformatik / Institute of Neuroinformatics