Boosting keyword spotting through on-device learnable user speech characteristics
METADATA ONLY
Loading...
Author / Producer
Date
2024-04
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Keyword spotting systems for always-on TinyML-constrained applications require on-site tuning to boost the accuracy of offline trained classifiers when deployed in unseen inference conditions. Adapting to the speech peculiarities of target users requires many in-domain samples, often unavailable in real-world scenarios. Furthermore, current on-device learning techniques rely on computationally intensive and memory-hungry backbone update schemes, unfit for always-on, battery-powered devices. In this work, we propose a novel on-device learning architecture, composed of a pretrained backbone and a user-aware embedding learning the user's speech characteristics. The so-generated features are fused and used to classify the input utterance. For domain shifts generated by unseen speakers, we measure error rate reductions of up to 19% from 30.1% to 24.3% based on the 35-class problem of the Google Speech Commands dataset, through the inexpensive update of the user projections. We moreover demonstrate the few-shot learning capabilities of our proposed architecture in sample- and class-scarce learning conditions. With 23.7 kparameters and 1 MFLOP per epoch required for on-device training, our system is feasible for TinyML applications aimed at battery-powered microcontrollers.
Permanent link
Publication status
published
Editor
Book title
Journal / series
Volume
Pages / Article No.
2403.07802
Publisher
Cornell University
Event
tinyML Research Symposium 2024
Edition / version
5 p.
Methods
Software
Geographic location
Date collected
Date created
Subject
Keyword Spotting; Embeddings; User Features; On-Device Learning
Organisational unit
03996 - Benini, Luca / Benini, Luca
Notes
Funding
207913 - TinyTrainer: On-chip Training for TinyML devices (SNF)