Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects
Abstract
Interactions with articulated objects are a challenging but important task for mobile robots. To tackle this challenge, we propose a novel closed-loop control pipeline, which integrates manipulation priors from affordance estimation with sampling-based whole-body control. We introduce the concept of agent-aware affordances which fully reflect the agent's capabilities and embodiment and we show that they outperform their state-of-the-art counterparts which are only conditioned on the end-effector geometry. Additionally, closed-loop affordance inference is found to allow the agent to divide a task into multiple non-continuous motions and recover from failure and unexpected states. Finally, the pipeline is able to perform long-horizon mobile manipulation tasks, i.e. opening and closing an oven, in the real world with high success rates (opening: 71%, closing: 72%). Show more
Publication status
publishedExternal links
Book title
2023 IEEE International Conference on Robotics and Automation (ICRA)Pages / Article No.
Publisher
IEEEEvent
Organisational unit
03737 - Siegwart, Roland Y. / Siegwart, Roland Y.
Funding
101017008 - Enhancing Healthcare with Assistive Robotic Mobile Manipulation (EC)
More
Show all metadata
ETH Bibliography
yes
Altmetrics