Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning
OPEN ACCESS
Author / Producer
Date
2020-12-11
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios. Functional decomposition between planning and low-level control is achieved by explicitly separating the state-action spaces across the hierarchy, which allows the integration of task-relevant knowledge per layer. We propose an RL-based planner to efficiently leverage the information in the planning layer of the hierarchy, while the control layer learns a goal-conditioned control policy. The hierarchy is trained jointly but allows for the composition of different policies such as transferring layers across multiple agents. We experimentally show that our method generalizes across unseen test environments and can scale to tasks well beyond 3x horizon length compared to both learning and non-learning based approaches. We evaluate on complex continuous control tasks with sparse rewards, including navigation and robot manipulation.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
Pages / Article No.
Publisher
Deep RL Workshop
Event
Workshop: Deep Reinforcement Learning @NeurIPS 2020 (virtual)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Hierarchical reinforcement learning; Robotics; Path planning; Reinforcement Learning
Organisational unit
03979 - Hilliges, Otmar (ehemalig) / Hilliges, Otmar (former)
Notes
Due to the Coronavirus (COVID-19) the conference was conducted virtually.
Funding
717054 - Optimization-based End-User Design of Interactive Technologies (EC)
Related publications and datasets
Is part of: