Show simple item record

dc.contributor.author
Gao, Dawei
dc.contributor.author
He, Xiaoxi
dc.contributor.author
Zhou, Zimu
dc.contributor.author
Tong, Yongxin
dc.contributor.author
Thiele, Lothar
dc.date.accessioned
2021-11-25T13:36:28Z
dc.date.available
2021-11-23T04:04:14Z
dc.date.available
2021-11-25T13:36:28Z
dc.date.issued
2021-10
dc.identifier.isbn
978-1-4503-8446-9
en_US
dc.identifier.other
10.1145/3459637.3482378
en_US
dc.identifier.uri
http://hdl.handle.net/20.500.11850/516416
dc.description.abstract
Adapting neural networks to unseen tasks with few training samples on resource-constrained devices benefits various Internet-of-Things applications. Such neural networks should learn the new tasks in few shots and be compact in size. Meta-learning enables few-shot learning, yet the meta-trained networks can be over-parameterised. However, naive combination of standard compression techniques like network pruning with meta-learning jeopardises the ability for fast adaptation. In this work, we propose adaptation-aware network pruning (ANP), a novel pruning scheme that works with existing meta-learning methods for a compact network capable of fast adaptation. ANP uses weight importance metric that is based on the sensitivity of the meta-objective rather than the conventional loss function, and adopts approximation of derivatives and layer-wise pruning techniques to reduce the overhead of computing the new importance metric. Evaluations on few-shot classification benchmarks show that ANP can prune meta-trained convolutional and residual networks by 85% without affecting their fast adaptation. © 2021 ACM
en_US
dc.language.iso
en
en_US
dc.publisher
Association for Computing Machinery
dc.subject
deep neural networks
en_US
dc.subject
meta learning
en_US
dc.subject
network pruning
en_US
dc.title
Pruning Meta-Trained Networks for On-Device Adaptation
en_US
dc.type
Conference Paper
dc.date.published
2021-10-26
ethz.book.title
Proceedings of the 30th ACM International Conference on Information & Knowledge Management
en_US
ethz.pages.start
514
en_US
ethz.pages.end
523
en_US
ethz.event
30th ACM International Conference on Information and Knowledge Management (CIKM '21)
en_US
ethz.event.location
Online
ethz.event.date
November 1-5, 2021
en_US
ethz.identifier.scopus
ethz.publication.place
New York, NY
ethz.publication.status
published
en_US
ethz.leitzahl
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02140 - Dep. Inf.technologie und Elektrotechnik / Dep. of Inform.Technol. Electrical Eng.::02640 - Inst. f. Technische Informatik und Komm. / Computer Eng. and Networks Lab.::03429 - Thiele, Lothar (emeritus) / Thiele, Lothar (emeritus)
ethz.leitzahl.certified
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02140 - Dep. Inf.technologie und Elektrotechnik / Dep. of Inform.Technol. Electrical Eng.::02640 - Inst. f. Technische Informatik und Komm. / Computer Eng. and Networks Lab.::03429 - Thiele, Lothar (emeritus) / Thiele, Lothar (emeritus)
ethz.date.deposited
2021-11-23T04:04:24Z
ethz.source
SCOPUS
ethz.eth
yes
en_US
ethz.availability
Metadata only
en_US
ethz.rosetta.installDate
2021-11-25T13:36:35Z
ethz.rosetta.lastUpdated
2024-02-02T15:26:56Z
ethz.rosetta.versionExported
true
ethz.COinS
ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.atitle=Pruning%20Meta-Trained%20Networks%20for%20On-Device%20Adaptation&rft.date=2021-10&rft.spage=514&rft.epage=523&rft.au=Gao,%20Dawei&He,%20Xiaoxi&Zhou,%20Zimu&Tong,%20Yongxin&Thiele,%20Lothar&rft.isbn=978-1-4503-8446-9&rft.genre=proceeding&rft_id=info:doi/10.1145/3459637.3482378&rft.btitle=Proceedings%20of%20the%2030th%20ACM%20International%20Conference%20on%20Information%20&%20Knowledge%20Management
 Search print copy at ETH Library

Files in this item

FilesSizeFormatOpen in viewer

There are no files associated with this item.

Publication type

Show simple item record