Riemannian Adaptive Optimization Methods
METADATA ONLY
Loading...
Author / Producer
Date
2023-05
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Several first order stochastic optimization methods commonly used in the Euclidean domain such as stochastic gradient descent (SGD), accelerated gradient descent or variance reduced methods have already been adapted to certain Riemannian settings. However, some of the most popular of these optimization tools - namely Adam , Adagrad and the more recent Amsgrad - remain to be generalized to Riemannian manifolds. We discuss the difficulty of generalizing such adaptive schemes to the most agnostic Riemannian setting, and then provide algorithms and convergence proofs for geodesically convex objectives in the particular case of a product of Riemannian manifolds, in which adaptivity is implemented across manifolds in the cartesian product. Our generalization is tight in the sense that choosing the Euclidean space as Riemannian manifold yields the same algorithms and regret bounds as those that were already known for the standard algorithms. Experimentally, we show faster convergence and to a lower train loss value for Riemannian adaptive methods over their corresponding baselines on the realistic task of embedding the WordNet taxonomy in the Poincare ball.
Permanent link
Publication status
published
External links
Editor
Book title
International Conference on Learning Representations (ICLR 2019)
Journal / series
Volume
9
Pages / Article No.
6384 - 6399
Publisher
Curran
Event
7th International Conference on Learning Representations (ICLR 2019)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
09462 - Hofmann, Thomas / Hofmann, Thomas
Notes
Conference lecture held on May 8, 2019.
Funding
Related publications and datasets
Is new version of: https://openreview.net/forum?id=r1eiqi09K7Is new version of: https://doi.org/10.48550/arXiv.1810.00760