Show simple item record

dc.contributor.author
Dubey, Rohit K.
dc.contributor.author
Sohn, Samuel S.
dc.contributor.author
Abualdenien, Jimmy
dc.contributor.author
Thrash, Tyler
dc.contributor.author
Hoelscher, Christoph
dc.contributor.author
Borrmann, André
dc.contributor.author
Kapadia, Mubbasir
dc.date.accessioned
2022-03-20T20:04:35Z
dc.date.available
2021-11-28T04:03:32Z
dc.date.available
2022-03-20T20:04:35Z
dc.date.issued
2021
dc.identifier.isbn
978-1-4503-9131-3
en_US
dc.identifier.other
10.1145/3487983.3488292
en_US
dc.identifier.uri
http://hdl.handle.net/20.500.11850/517374
dc.description.abstract
Reinforcement learning (RL) has demonstrated great success in solving navigation tasks but often fails when learning complex environmental structures. One open challenge is to incorporate low-level generalizable skills with human-like adaptive path-planning in an RL framework. Motivated by neural findings in animal navigation, we propose a Successor eNtropy-based Adaptive Path-planning (SNAP) that combines a low-level goal-conditioned policy with the flexibility of a classical high-level planner. SNAP decomposes distant goal-reaching tasks into multiple nearby goal-reaching sub-tasks using a topological graph. To construct this graph, we propose an incremental subgoal discovery method that leverages the highest-entropy states in the learned Successor Representation. The Successor Representation encodes the likelihood of being in a future state given the current state and capture the relational structure of states based on a policy. Our main contributions lie in discovering subgoal states that efficiently abstract the state-space and proposing a low-level goal-conditioned controller for local navigation. Since the basic low-level skill is learned independent of state representation, our model easily generalizes to novel environments without intensive relearning. We provide empirical evidence that the proposed method enables agents to perform long-horizon sparse reward tasks quickly, take detours during barrier tasks, and exploit shortcuts that did not exist during training. Our experiments further show that the proposed method outperforms the existing goal-conditioned RL algorithms in successfully reaching distant-goal tasks and policy learning. To evaluate human-like adaptive path-planning, we also compare our optimal agent with human data and found that, on average, the agent was able to find a shorter path than the human participants.
en_US
dc.language.iso
en
en_US
dc.publisher
Association for Computing Machinery
en_US
dc.subject
Goal-conditioned RL
en_US
dc.subject
Robot navigation
en_US
dc.subject
Option discovery
en_US
dc.subject
Adaptive path-planning
en_US
dc.subject
Hippocampus
en_US
dc.title
SNAP:Successor Entropy based Incremental Subgoal Discovery for Adaptive Navigation
en_US
dc.type
Conference Paper
dc.date.published
2021-11-10
ethz.book.title
MIG '21: Motion, Interaction and Games
en_US
ethz.pages.start
16
en_US
ethz.size
11 p.
en_US
ethz.event
14th ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG 2021)
en_US
ethz.event.location
Online
en_US
ethz.event.date
November 10 - 12, 2021
en_US
ethz.identifier.scopus
ethz.publication.place
New York, NY
en_US
ethz.publication.status
published
en_US
ethz.date.deposited
2021-11-28T04:03:50Z
ethz.source
SCOPUS
ethz.eth
yes
en_US
ethz.availability
Metadata only
en_US
ethz.rosetta.installDate
2022-03-20T20:04:58Z
ethz.rosetta.lastUpdated
2022-03-20T20:04:58Z
ethz.rosetta.versionExported
true
ethz.COinS
ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.atitle=SNAP:Successor%20Entropy%20based%20Incremental%20Subgoal%20Discovery%20for%20Adaptive%20Navigation&rft.date=2021&rft.spage=16&rft.au=Dubey,%20Rohit%20K.&Sohn,%20Samuel%20S.&Abualdenien,%20Jimmy&Thrash,%20Tyler&Hoelscher,%20Christoph&rft.isbn=978-1-4503-9131-3&rft.genre=proceeding&rft_id=info:doi/10.1145/3487983.3488292&rft.btitle=MIG%20'21:%20Motion,%20Interaction%20and%20Games
 Search print copy at ETH Library

Files in this item

FilesSizeFormatOpen in viewer

There are no files associated with this item.

Publication type

Show simple item record