Metadata only
Author
Show all
Date
2020Type
- Conference Paper
Citations
Cited 11 times in
Web of Science
Cited 13 times in
Scopus
ETH Bibliography
yes
Altmetrics
Abstract
Low bit-width Quantized Neural Networks (QNNs) enable deployment of complex machine learning models on constrained devices such as microcontrollers (MCUs) by reducing their memory footprint. Fine-grained asymmetric quantization (i.e., different bit-widths assigned to weights and activations on a tensor-by-tensor basis) is a particularly interesting scheme to maximize accuracy under a tight memory constraint. However, the lack of sub-byte instruction set architecture (ISA) support in SoA microprocessors makes it hard to fully exploit this extreme quantization paradigm in embedded MCUs. Support for sub-byte and asymmetric QNNs would require many precision formats and an exorbitant amount of opcode space. In this work, we attack this problem with status-based SIMD instructions: rather than encoding precision explicitly, each operand's precision is set dynamically in a core status register. We propose a novel RISC-V ISA core MPIC (Mixed Precision Inference Core) based on the open-source RI5CY core. Our approach enables full support for mixed-precision QNN inference with 292 different combinations of operands at 16-, 8-, 4- and 2-bit precision, without adding any extra opcode or increasing the complexity of the decode stage. Our results show that MPIC improves both performance and energy efficiency by a factor of 1.1-4.9x when compared to software-based mixed-precision on RI5CY; with respect to commercially available Cortex-M4 and M7 microcontrollers, it delivers 3.6-11.7x better performance and 41-155x higher efficiency. © 2020 IEEE. Show more
Publication status
publishedExternal links
Book title
2020 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)Pages / Article No.
Publisher
IEEEEvent
Subject
PULP Platform; Embedded-Systems; Deep Neural Networks; Mixed-precision; MicrocontrollerOrganisational unit
03996 - Benini, Luca / Benini, Luca
Funding
732631 - Open Transprecision Computing (EC)
Notes
Due to the Corona virus (COVID-19) the conference was conducted virtually.More
Show all metadata
Citations
Cited 11 times in
Web of Science
Cited 13 times in
Scopus
ETH Bibliography
yes
Altmetrics