Neural Oscillators are Universal
METADATA ONLY
Loading...
Author / Producer
Date
2023-05
Publication Type
Report
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Coupled oscillators are being increasingly used as the basis of machine learning (ML) architectures, for instance in sequence modeling, graph representation learning and in physical neural networks that are used in analog ML devices. We introduce an abstract class of neural oscillators that encompasses these architectures and prove that neural oscillators are universal, i.e, they can approximate any continuous and casual operator mapping between time-varying functions, to desired accuracy. This universality result provides theoretical justification for the use of oscillator based ML systems. The proof builds on a fundamental result of independent interest, which shows that a combination of forced harmonic oscillators with a nonlinear read-out suffices to approximate the underlying operators.
Permanent link
Publication status
published
Editor
Book title
Journal / series
Volume
2023-20
Pages / Article No.
Publisher
Seminar for Applied Mathematics, ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Neural oscillators; Neural ODEs; Universal approximation; Deep learning; Hamiltonian systems
Organisational unit
03851 - Mishra, Siddhartha / Mishra, Siddhartha
Notes
Funding
770880 - Computation and analysis of statistical solutions of fluid flow (EC)
Related publications and datasets
Is previous version of: