Metadata only
Date
2022Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
Ensuring safety is a crucial challenge when deploying reinforcement learning (RL) to real-world systems. We develop confidence-based safety filters, a control-theoretic approach for certifying state safety constraints for nominal policies learnt via standard RL techniques, based on probabilistic dynamics models. Our approach is based on a reformulation of state constraints in terms of cost functions, reducing safety verification to a standard RL task. By exploiting the concept of hallucinating inputs, we extend this formulation to determine a "backup" policy which is safe for the unknown system with high probability. The nominal policy is minimally adjusted at every time step during a roll-out towards the backup policy, such that safe recovery can be guaranteed afterwards. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach. Show more
Publication status
publishedExternal links
Book title
2022 IEEE 61st Conference on Decision and Control (CDC)Pages / Article No.
Publisher
IEEEEvent
Funding
815943 - Reliable Data-Driven Decision Making in Cyber-Physical Systems (EC)
180545 - NCCR Automation (phase I) (SNF)
More
Show all metadata
ETH Bibliography
yes
Altmetrics