Learning Functionally Decomposed Hierarchies for Continuous Control Tasks With Path Planning
METADATA ONLY
Loading...
Author / Producer
Date
2021-04
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios. Functional decomposition between planning and low-level control is achieved by explicitly separating the state-action spaces across the hierarchy, which allows the integration of task-relevant knowledge per layer. We propose an RL-based planner to efficiently leverage the information in the planning layer of the hierarchy, while the control layer learns a goal-conditioned control policy. The hierarchy is trained jointly but allows for the modular transfer of policy layers across hierarchies of different agents. We experimentally show that our method generalizes across unseen test environments and can scale to 3x horizon length compared to both learning and non-learning based methods. We evaluate on complex continuous control tasks with sparse rewards, including navigation and robot manipulation. © 2021 IEEE
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
6 (2)
Pages / Article No.
3623 - 3630
Publisher
IEEE
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Hierarchical reinforcement learning; Motion and path planning; Reinforcement learning; Transfer learning
Organisational unit
03979 - Hilliges, Otmar (ehemalig) / Hilliges, Otmar (former)
Notes
Funding
717054 - Optimization-based End-User Design of Interactive Technologies (EC)