Neuroevolution of a Modular Memory-Augmented Neural Network for Deep Memory Problems
METADATA ONLY
Loading...
Author / Creator
Date
2019
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
We present Modular Memory Units (MMUs), a new class of memory-augmented neural network. MMU builds on the gated neural architecture of Gated Recurrent Units (GRUs) and Long Short Term Memory (LSTMs), to incorporate an external memory block, similar to a Neural Turing Machine (NTM). MMU interacts with the memory block using independent read and write gates that serve to decouple the memory from the central feedforward operation. This allows for regimented memory access and update, giving our network the ability to choose when to read from memory, update it, or simply ignore it. This capacity to act in detachment allows the network to shield the memory from noise and other distractions, while simultaneously using it to effectively retain and propagate information over an extended period of time. We train MMU using both neuroevolution and gradient descent, and perform experiments on two deep memory benchmarks. Results demonstrate that MMU performs significantly faster and more accurately than traditional LSTM-based methods, and is robust to dramatic increases in the sequence depth of these memory benchmarks.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
27 (4)
Pages / Article No.
639 - 664
Publisher
MIT Press
Event
Edition / version
Methods
Geographic location
Date collected
Date created
Subject
RNNS; memory-augmented neural networks; neuroevolution