Hedge Algorithm and Dual Averaging schemes
OPEN ACCESS
Loading...
Author / Producer
Date
2013-06
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
We show that the Hedge algorithm, a method that is widely used in Machine Learning, can be interpreted as a particular instance of Dual Averaging schemes, which have recently been introduced by Nesterov for regret minimization. Based on this interpretation, we establish three alternative methods of the Hedge algorithm: one in the form of the original method, but with optimal parameters, one that requires less a priori information, and one that is better adapted to the context of the Hedge algorithm. All our modified methods have convergence results that are better or at least as good as the performance guarantees of the vanilla method. In numerical experiments, our methods significantly outperform the original scheme.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
77 (3)
Pages / Article No.
279 - 289
Publisher
Springer
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
First-order methods; Hedge algorithm; Dual Averaging methods; Convex optimization
Organisational unit
03873 - Weismantel, Robert / Weismantel, Robert
Notes
It was possible to publish this article open access thanks to a Swiss National Licence with the publisher