Metadata only
Date
2023Type
- Journal Article
ETH Bibliography
yes
Altmetrics
Abstract
This letter is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness guarantees. The proposed scheme is based on notions of stochastic bisimulation functions (SBF) to capture the probabilistic distance between state trajectories of an unknown stochastic system and those of finite MDP. In our proposed setting, we first reformulate corresponding conditions of SBF as a robust convex program (RCP). We then propose a scenario convex program (SCP) associated to the original RCP by collecting a finite number of data from trajectories of the system. We ultimately construct an SBF between the data-driven finite MDP and the unknown stochastic system with a given confidence level by establishing a probabilistic relation between optimal values of the SCP and the RCP. We also propose two different approaches for the construction of finite MDPs from data. We illustrate the efficacy of our results over a nonlinear jet engine compressor with unknown dynamics. We construct a data-driven finite MDP as a suitable substitute of the original system to synthesize controllers maintaining the system in a safe set with some probability of satisfaction and a desirable confidence level. Show more
Publication status
publishedExternal links
Journal / series
IEEE Control Systems LettersVolume
Pages / Article No.
Publisher
IEEESubject
Data-driven synthesis; MDP abstractions; Stochastic bisimulation functions; Formal guaranteesOrganisational unit
09574 - Frazzoli, Emilio / Frazzoli, Emilio
More
Show all metadata
ETH Bibliography
yes
Altmetrics