Identifying Terrain Physical Parameters from Vision - Towards Physical-Parameter-Aware Locomotion and Navigation
OPEN ACCESS
Loading...
Author / Producer
Date
2024-11
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Identifying the physical properties of the surround ing environment is essential for robotic locomotion and naviga tion to deal with non-geometric hazards, such as slippery and deformable terrains. It would be of great benefit for robots to anticipate these extreme physical properties before contact; however, estimating environmental physical parameters from vision is still an open challenge. Animals can achieve this by using their prior experience and knowledge of what they have seen and how it felt. In this work, we propose a cross-modal self supervised learning framework for vision-based environmental physical parameter estimation, which paves the way for future physical-property-aware locomotion and navigation. We bridge the gap between existing policies trained in simulation and identification of physical terrain parameters from vision. We propose to train a physical decoder in simulation to predict fric tion and stiffness from multi-modal input. The trained network allows the labeling of real-world images with physical parameters in a self-supervised manner to further train a visual network during deployment, which can densely predict the friction and stiffness from image data. We validate our physical decoder in simulation and the real world using a quadruped ANYmal robot, outperforming an existing baseline method. We show that our visual network can predict the physical properties in indoor and outdoor experiments while allowing fast adaptation to new environments. — Project Page https://bit.ly/3Xo5AA8
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
9 (11)
Pages / Article No.
9279 - 9286
Publisher
IEEE
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Legged robots; Deep learning for visual perception; Field robots
Organisational unit
09570 - Hutter, Marco / Hutter, Marco
Notes
Funding
166232 - Data-driven control approaches for advanced legged locomotion (SNF)
188596 - Perceptive Dynamic Locomotion on Rough Terrain (SNF)
101016970 - Natural Intelligence for Robotic Monitoring of Habitats (EC)
188596 - Perceptive Dynamic Locomotion on Rough Terrain (SNF)
101016970 - Natural Intelligence for Robotic Monitoring of Habitats (EC)