Recurrent neural network approximation theory
OPEN ACCESS
Loading...
Author / Producer
Date
2025
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
In this thesis we study the approximation capabilities of recurrent neural networks (RNNs). Firstly, we consider the approximation of certain classes of systems mapping an input sequence to an output sequence. We prove that RNNs can approximate any such system to within arbitrarily small worst-case error. What is more, we derive quantitative results on the amount of information needed to specify the approximating RNN. Furthermore, we present a framework to unify the study of different approximation tasks, allowing us to conclude that neural network learning is universally information-optimal for a large variety of approximation problems throughout both function and system approximation.
Secondly, we study the use of RNNs for approximating real-valued polynomials. We find that for every polynomial there exists an RNN that produces — as its output sequence — increasingly precise approximations to the target polynomial. This is remarkable as the weights of the RNN do not depend on the desired approximation error. That is, any arbitrarily small approximation error is achieved simply by running the approximating RNN for more time steps.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner: Bölcskei, Helmut
Examiner : Petersen, Philipp
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Neural networks; Approximation Theory; Machine learning
Organisational unit
03610 - Boelcskei, Helmut / Boelcskei, Helmut