Show simple item record

dc.contributor.author
Li, Hao
dc.contributor.author
De, Soham
dc.contributor.author
Xu, Zheng
dc.contributor.author
Studer, Christoph
dc.contributor.author
Samet, Hanan
dc.contributor.author
Goldstein, Tom
dc.contributor.editor
von Luxburg, Ulrike
dc.contributor.editor
Guyon, Isabelle
dc.contributor.editor
Bengio, Samy
dc.contributor.editor
Wallach, Hanna M.
dc.contributor.editor
Fergus, Rob
dc.contributor.editor
Vishwanathan, S.V.N.
dc.contributor.editor
Garnett, Roman
dc.date.accessioned
2021-03-05T14:34:47Z
dc.date.available
2021-01-07T17:08:00Z
dc.date.available
2021-03-05T14:34:47Z
dc.date.issued
2018
dc.identifier.uri
http://hdl.handle.net/20.500.11850/460351
dc.description.abstract
Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-precision model for efficient inference on such systems. However, training models directly with coarsely quantized weights is a key step towards learning on embedded platforms that have limited computing resources, memory capacity, and power consumption. Numerous recent publications have studied methods for training quantized networks, but these studies have mostly been empirical. In this work, we investigate training methods for quantized neural networks from a theoretical viewpoint. We first explore accuracy guarantees for training methods under convexity assumptions. We then look at the behavior of these algorithms for non-convex problems, and show that training algorithms that exploit high-precision representations have an important greedy search phase that purely quantized training methods lack, which explains the difficulty of training using low-precision arithmetic.
en_US
dc.language.iso
en
en_US
dc.publisher
Curran
en_US
dc.title
Training Quantized Nets: A Deeper Understanding
en_US
dc.type
Conference Paper
ethz.book.title
Advances in Neural Information Processing Systems 30
en_US
ethz.journal.volume
9
en_US
ethz.pages.end
5812
en_US
ethz.size
5822
en_US
ethz.event
31st Annual Conference on Neural Information Processing Systems (NIPS 2017)
en_US
ethz.event.location
Long Beach, CA, USA
en_US
ethz.event.date
December 4-9, 2017
en_US
ethz.publication.place
Red Hook, NY
en_US
ethz.publication.status
published
en_US
ethz.leitzahl
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02140 - Dep. Inf.technologie und Elektrotechnik / Dep. of Inform.Technol. Electrical Eng.::02636 - Institut für Integrierte Systeme / Integrated Systems Laboratory::09695 - Studer, Christoph / Studer, Christoph
en_US
ethz.leitzahl.certified
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02140 - Dep. Inf.technologie und Elektrotechnik / Dep. of Inform.Technol. Electrical Eng.::02636 - Institut für Integrierte Systeme / Integrated Systems Laboratory::09695 - Studer, Christoph / Studer, Christoph
en_US
ethz.identifier.url
https://papers.nips.cc/paper/2017/hash/1c303b0eed3133200cf715285011b4e4-Abstract.html
ethz.date.deposited
2021-01-07T17:08:11Z
ethz.source
FORM
ethz.eth
no
en_US
ethz.availability
Metadata only
en_US
ethz.rosetta.installDate
2021-03-05T14:34:58Z
ethz.rosetta.lastUpdated
2021-03-05T14:34:58Z
ethz.rosetta.versionExported
true
ethz.COinS
ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.atitle=Training%20Quantized%20Nets:%20A%20Deeper%20Understanding&rft.date=2018&rft.volume=9&rft.epage=5812&rft.au=Li,%20Hao&De,%20Soham&Xu,%20Zheng&Studer,%20Christoph&Samet,%20Hanan&rft.genre=proceeding&rft.btitle=Advances%20in%20Neural%20Information%20Processing%20Systems%2030
 Search print copy at ETH Library

Files in this item

FilesSizeFormatOpen in viewer

There are no files associated with this item.

Publication type

Show simple item record