Towards trustworthy AI: Safe-visor architecture for uncertified controllers in stochastic cyber-physical systems
- Conference Paper
Artificial intelligence-based (a.k.a. AI-based) controllers have received significant attentions in the past few years due to their broad applications in cyber-physical systems (CPSs) to accomplish complex control missions. However, guaranteeing safety and reliability of CPSs equipped with this kind of (uncertified) controllers is currently very challenging, which is of vital importance in many real-life safety-critical applications. To cope with this difficulty, we propose a Safe-visor architecture for sandboxing AI-based controllers in stochastic CPSs. The proposed framework contains (i) a history-based supervisor which checks inputs from the AI-based controller and makes compromise between functionality and safety of the system, and (ii) a safety advisor that provides fallback when the AI-based controller endangers the safety of the system. By employing this architecture, we provide formal probabilistic guarantees on the satisfaction of those classes of safety specifications which can be represented by the accepting languages of deterministic finite automata (DFA), while AI-based controllers can still be employed in the control loop even though they are not reliable. Show more
Book titleProceedings of the Workshop on Computation-Aware Algorithmic Design for Cyber-Physical Systems (CAADCPS '21)
Pages / Article No.
PublisherAssociation for Computing Machinery
SubjectTrustworthy AI; Safe-visor architecture; AI-based controllers; Stochastic cyber-physical systems; Artificial intelligence
NotesA satellite workshop of the 2021 CPS-IoT week, May 18-21, 2021.
MoreShow all metadata