Exploring Constrained Reinforcement Learning Algorithms for Quadrupedal Locomotion
OPEN ACCESS
Loading...
Author / Producer
Date
2024
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Shifting from traditional control strategies to Deep Reinforcement Learning (RL) for legged robots poses inherent challenges, especially when addressing real-world physical con straints during training. While high-fidelity simulations provide significant benefits, they often bypass these essential physical limitations. In this paper, we experiment with the Constrained Markov Decision Process (CMDP) framework instead of the conventional unconstrained RL for robotic applications. We evaluated five constrained policy optimization algorithms for quadrupedal locomotion using three different robot models. Our aim is to evaluate their applicability in real-world sce narios. Our robot experiments demonstrate the critical role of incorporating physical constraints, yielding successful sim-to-real transfers, and reducing operational errors on physical systems. The CMDP formulation streamlines the training process by separately handling constraints from rewards. Our findings underscore the potential of constrained RL for the effective development and deployment of learned controllers in robotics.
Permanent link
Publication status
published
Editor
Book title
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Journal / series
Volume
Pages / Article No.
11132 - 11138
Publisher
IEEE
Event
37th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
09570 - Hutter, Marco / Hutter, Marco
Notes
Funding
852044 - Learning Mobility for Real Legged Robots (EC)