Architectural Considerations for Deep Reinforcement Learning
OPEN ACCESS
Author / Producer
Date
2022
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Since the advent of deep learning and deep reinforcement learning, there have been numerous empirical successes that employ some sort of artificial neural network for a given optimization problem. However, many of the inner workings are only vaguely understood and hidden among the success stories. By uncovering various implications, this thesis tries to establish an understanding of why certain neural network architecture designs work, and crucially why others do not. Instead of focusing on empirical results, this thesis starts by investigating simple mathematical implications that result from architecture design and training with backpropagation. It then proposes a sparse alternative to fully connected layers which avoids bottlenecks in the signal propagation. It further shows how to design monotonic neural networks and how these can be used to give agents a more flexible policy representation in a continuous action space control setup. The book further discusses trade-offs and designs for splitting neural networks into multiple modules. In particular, the need for modularization is exemplified in multi-task setups with conflicting objectives. Finally, the thesis discusses recently proposed attention architectures and their hidden implications. The results throughout the thesis highlight correlated effects between hyperparameters and the necessity for tailored architecture design. This thesis is suitable for readers with a technical background. It is written such that it can be understood by people who are just getting started with neural networks. However, even a senior researcher in the field might find interest in the unique viewpoints that are presented.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Wattenhofer, Roger
Examiner : Kim, Kee-Eung
Examiner : Wang, Donglin
Examiner : Dabney, Will
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Deep Learning; Reinforcement Learning; Neural network architecture; Multi-task learning
Organisational unit
03604 - Wattenhofer, Roger / Wattenhofer, Roger