Safety, Stability and Performance in Learning-Based Control Using Predictive Control Methods
OPEN ACCESS
Loading...
Author / Producer
Date
2025
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
The optimal control problem is a fundamental concept in control theory. The aim is to minimize a cost function subject to the modeled dynamics of the system. Many applications in the domains of robotics, autonomous driving, aerospace and biomedical engineering among others fall within this framework. Beyond performance specifications, most real-world systems have physical or task-specific restrictions, which should be satisfied during operation. These restrictions can be formulated mathematically as constraints in the optimal control problem. Additionally, models are generally not accurate and thereby model uncertainties such as exogenous disturbances need to be considered, leading to a constrained robust optimal control problem (CROCP) formulation.
Popular approaches to obtain approximate solutions to optimal control problems are given by model predictive control (MPC) and learning-based methods, such as, e.g., reinforcement learning. While MPC provides a principled solution approach when the system is subject to constraints, it can struggle in tasks with sparse cost signals which require a long prediction horizon. Learning-based methods can overcome these issues, however it can be difficult to ensure constraint satisfaction guarantees, especially when the system is subject to exogenous disturbances. Safety filters have recently emerged as a principled method to augment controllers with constraint satisfaction guarantees. The proposed inputs are projected onto the set of inputs for which constraints can be satisfied for all times, aiming to retain the performance obtained from the employed controller, while ensuring safe system operation. If the constraints on the system are ever violated during operation, methods should ideally be designed to ensure that the system recovers back to a safe region of operation. A framework that provides such guarantees is given by discrete-time control barrier functions (CBF). CBFs extend Lyapunov functions, allowing to certify invariance and stability of a set, rather than the origin, in the state space. However, similarly to Lyapunov functions, obtaining an explicit formulation of a CBF can be a challenging task.
A key challenge is therefore to design control methods, that provide robust constraint satisfaction, ideally with recovery guarantees, while achieving a high performance. Such methods should introduce little conservativeness in terms of overly restricting the system from approaching constraints, while at the same time being computationally efficient. In this thesis, we consider approaches which aim to satisfy these requirements, either solving the CROCP directly or augmenting any controller with robust constraint satisfaction and recovery guarantees.
In the first part of the thesis, we consider a specific cost function for the optimal control problem, i.e., we consider the generalized dynamic regret, a comparative performance metric. For systems subject to exogenous disturbances, the controller which achieves the optimal cost has access to all future disturbances in a non-causal fashion. As this is generally not implementable for dynamical systems, regret optimal control aims to minimize the cost difference to this surrogate benchmark controller. We propose a semi-definite program (SDP) for the generalized dynamic regret minimization problem for linear dynamical systems subject to additive disturbances with bounded energy. If explicit bounds on the disturbance are known at every time step, we modify the proposed method such that an improved bound on the incurred regret can be achieved. The optimization problem formulation enables integrating state and input constraints in the controller synthesis, allowing to compute a controller with closed-loop constraint satisfaction guarantees.
In the second part of the thesis, we consider predictive control methods, which guarantee constraint satisfaction for nonlinear dynamical systems subject to disturbances. Additionally, we provide recovery mechanisms in case the constraints are violated during operation due to unexpected disturbances acting on the system. The proposed methods allow for the integration of a general control cost or they can be framed in terms of a safety filter, allowing to augment learning-based control methods. First, we consider how state measurements during online operation can be used to reduce conservativeness of predictive safety filters by improving uncertain model descriptions and improving its terminal set. Next, we provide a theoretical analysis of discrete-time CBFs allowing to analyze closed-loop behavior when the system is subject to disturbances and propose a robust MPC-based CBF formulation. We propose multiobjective approaches allowing to guarantee closed-loop stability through a Lyapunov and CBF decrease constraint, while optimizing a primary performance objective. Finally, we propose a computationally efficient approximation-based CBF formulation.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Zeilinger , Melanie N.
Examiner : Ferrari Trecate, Giancarlo
Examiner : Sreenath , Koushil
Examiner : Carron, Andrea
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
09563 - Zeilinger, Melanie / Zeilinger, Melanie