- Conference Paper
Adapting neural networks to unseen tasks with few training samples on resource-constrained devices benefits various Internet-of-Things applications. Such neural networks should learn the new tasks in few shots and be compact in size. Meta-learning enables few-shot learning, yet the meta-trained networks can be over-parameterised. However, naive combination of standard compression techniques like network pruning with meta-learning jeopardises the ability for fast adaptation. In this work, we propose adaptation-aware network pruning (ANP), a novel pruning scheme that works with existing meta-learning methods for a compact network capable of fast adaptation. ANP uses weight importance metric that is based on the sensitivity of the meta-objective rather than the conventional loss function, and adopts approximation of derivatives and layer-wise pruning techniques to reduce the overhead of computing the new importance metric. Evaluations on few-shot classification benchmarks show that ANP can prune meta-trained convolutional and residual networks by 85% without affecting their fast adaptation. © 2021 ACM Show more
Book titleProceedings of the 30th ACM International Conference on Information & Knowledge Management
Pages / Article No.
PublisherAssociation for Computing Machinery
Subjectdeep neural networks; meta learning; network pruning
Organisational unit03429 - Thiele, Lothar (emeritus) / Thiele, Lothar (emeritus)
MoreShow all metadata