Online Performance Optimization of Nonlinear Systems: A Gray-Box Approach
OPEN ACCESS
Loading...
Author / Producer
Date
2024-07
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
We propose a gray-box controller to optimize the performance of a nonlinear system in an online manner. This is motivated by the observation that model-based and model-free approaches own complementary benefits in sample efficiency and optimality in the presence of inaccurate models. To achieve the best of both worlds, our controller incorporates approximate model information into model-free updates via adaptive convex combinations. Further, it leverages real-time outputs of the system and iteratively adjusts control inputs. We quantify conditions on the quality of approximate models that render the gray-box approach preferable to model-based or model-free approaches. We characterize the performance of our controller via dynamic regret in a constrained, time-varying setting, and highlight how the regret scales with the number of iterations, the problem dimension, and the cumulative effect of inaccurate models.
Permanent link
Publication status
published
External links
Editor
Book title
ICML 2024 Workshop: Foundations of Reinforcement Learning and Control - Connections and Perspectives
Journal / series
Volume
Pages / Article No.
Publisher
OpenReview
Event
Workshop on Foundations of Reinforcement Learning and Control: Connections and Perspectives (FoRLac 2024)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
09478 - Dörfler, Florian / Dörfler, Florian
02650 - Institut für Automatik / Automatic Control Laboratory
Notes
Held in conjunction with the 41st International Conference on Machine Learning (ICML 2024) ; Poster presentation on July 26, 2024