Suche
Ergebnisse
-
Spectral Tensor Train Parameterization of Deep Learning Layers
(2021)Proceedings of Machine Learning Research ~ Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. The low-rank property leads to parameter efficiency and permits taking computational shortcuts when computing mappings. Spectral properties are often subject to constraints in optimization problems, leading to better models and stability of optimization. We start by looking at the compact SVD parameterization of weight ...Conference Paper