IEEE Control Systems Letters

Journal: IEEE Control Systems Letters

Publisher

IEEE

ISSN

2475-1456

Show all metadata

Search Results

Publications 1 - 10 of 72

Constructing MDP Abstractions Using Data with Formal Guarantees
Lavaei, Abolfazl; Soudjani, Sadegh; Frazzoli, Emilio; et al. (2023)
IEEE Control Systems Letters
This letter is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness guarantees. The proposed scheme is based on notions of stochastic bisimulation functions (SBF) to capture the probabilistic distance between state trajectories of an unknown stochastic system and those of finite MDP. In our proposed setting, we first reformulate corresponding conditions of SBF as a robust convex program (RCP). We then propose a scenario convex program (SCP) associated to the original RCP by collecting a finite number of data from trajectories of the system. We ultimately construct an SBF between the data-driven finite MDP and the unknown stochastic system with a given confidence level by establishing a probabilistic relation between optimal values of the SCP and the RCP. We also propose two different approaches for the construction of finite MDPs from data. We illustrate the efficacy of our results over a nonlinear jet engine compressor with unknown dynamics. We construct a data-driven finite MDP as a suitable substitute of the original system to synthesize controllers maintaining the system in a safe set with some probability of satisfaction and a desirable confidence level.
Inverse Optimal Control with Constraint Relaxation
Rickenbach, Rahel; Lahr, Amon; Zeilinger, Melanie N. (2025)
IEEE Control Systems Letters
Inverse optimal control (IOC) is a promising paradigm for learning and mimicking optimal control strategies from capable demonstrators, or gaining a deeper understanding of their intentions, by estimating an unknown objective function from one or more corresponding optimal control sequences. When computing estimates from demonstrations in environments with safety-preserving inequality constraints, acknowledging their presence in the chosen IOC method is crucial given their strong influence on the final control strategy. However, solution strategies capable of considering inequality constraints, such as the inverse Karush-Kuhn-Tucker approach, rely on their correct activation and fulfillment; a restrictive assumption when dealing with noisy demonstrations. To overcome this problem, we leverage the concept of exact penalty functions for IOC and show preservation of estimation accuracy. Considering noisy demonstrations, we then illustrate how the usage of penalty functions reduces the number of unknown variables and how their approximations enhance the estimation method’s capacity to account for wrong constraint activations within a polytopic-constrained environment. The proposed method is evaluated for three systems in simulation, outperforming traditional relaxation approaches for noisy demonstrations.
A Soft Constrained MPC Formulation Enabling Learning From Trajectories With Constraint Violations
Wabersich, Kim P.; Krishnadas, Raamadaas; Zeilinger, Melanie N. (2021)
IEEE Control Systems Letters
In practical model predictive control (MPC) implementations, constraints on the states are typically softened to ensure feasibility despite unmodeled disturbances. In this work, we propose a soft constrained MPC formulation supporting polytopic terminal sets in half-space and vertex representation, which significantly increases the feasible set while maintaining asymptotic stability in case of constraint violations. The proposed formulation allows for leveraging system trajectories that violate state constraints to iteratively improve the MPC controller’s performance. To this end, we apply convex optimization techniques to obtain a data-driven terminal cost and set, which result in a quadratic MPC problem.
A Fast Method for Real-Time Chance-Constrained Decision with Application to Power Systems
Bolognani, Saverio; Arcari, Elena; Dörfler, Florian (2017)
IEEE Control Systems Letters
Estimation of Infinitesimal Generators for Unknown Stochastic Hybrid Systems via Sampling: A Formal Approach
Nejati, Ameneh; Lavaei, Abolfazl; Soudjani, Sadegh; et al. (2022)
IEEE Control Systems Letters
In this letter, we develop a data-driven framework with formal confidence bounds for the estimation of infinitesimal generators for continuous-time stochastic hybrid systems with unknown dynamics. The proposed approximation scheme employs both time discretization and sampling from the solution process, and estimates the infinitesimal generator of the solution process via a set of data collected from trajectories of systems. We assume some mild continuity assumptions on the dynamics of the system and quantify the closeness between the infinitesimal generator and its approximation while ensuring an a-priori guaranteed confidence bound. To provide a reasonable closeness precision, we discuss significant roles of both time discretization and number of data in our approximation scheme. In particular, for a fixed number of data, variance of the estimation converges to infinity when the time discretization goes to zero. The proposed approximation framework guides us how to jointly select a suitable data size and a time discretization parameter to cope with this counter-intuitive behavior. We demonstrate the effectiveness of our proposed results by applying them to a nonlinear jet engine compressor with unknown dynamics.
Asymptotically exact unweighted particle filter for manifold-valued hidden states and point process observations
Surace, Simone C.; Kutschireiter, Anna; Pfister, Jean-Pascal (2020)
IEEE Control Systems Letters
The filtering of a Markov diffusion process on a manifold from counting process observations leads to ‘large’ changes in the conditional distribution upon an observed event, corresponding to a multiplication of the density by the intensity function of the observation process. If that distribution is represented by unweighted samples or particles, they need to be jointly transformed such that they sample from the modified distribution. In previous work, this transformation has been approximated by a translation of all the particles by a common vector. However, such an operation is ill-defined on a manifold, and on a vector space, a constant gain can lead to a wrong estimate of the uncertainty over the hidden state. Here, taking inspiration from the feedback particle filter (FPF), we derive an asymptotically exact filter (called ppFPF) for point process observations, whose particles evolve according to intrinsic (i.e., parametrization-invariant) dynamics that are composed of the dynamics of the hidden state plus additional control terms. While not sharing the gain-times-error structure of the FPF, the optimal control terms are expressed as solutions to partial differential equations analogous to the weighted Poisson equation for the gain of the FPF. The proposed filter can therefore make use of existing approximation algorithms for solutions of weighted Poisson equations.
On Decentralized Stability Conditions Using Scaled Relative Graphs
Baron-Prada, Eder; Anta, Adolfo; Dörfler, Florian (2025)
IEEE Control Systems Letters
Traditional centralized methods for stability analysis in linear multi-agent systems face significant challenges, including limited scalability, lack of modularity, and difficulties in distributed implementation. Various decentralized approaches have been developed to overcome these issues, such as passivity-based techniques and methods combining small gain and phase theorems. Although these approaches improve scalability, they often yield conservative results. In this letter, we introduce a novel set of decentralized stability conditions based on the Scaled Relative Graphs (SRG) framework, providing an efficient and effective tool for analyzing large-scale systems.
Noncooperative Games With Prospect Theoretic Preferences
Fochesato, Marta; Pokou, Frédy; Le Cadre, Hélène; et al. (2025)
IEEE Control Systems Letters
We study noncooperative games with uncertain payoffs, where agents display irrational behaviors in response to underlying risk factors. Our formulation incorporates prospect theory, a behavioral model used to describe agents' risk attitude, into the equilibrium theory of noncooperative $N-$ agent games. We show that the resulting nonconvex nonsmooth game admits equilibria and provide convergence guarantees for their computation. Further, the concept of "Price of Irrationality" is introduced to quantify the suboptimality induced by irrational behaviors. Finally, we provide bounds on the performance of a class of prospect theoretic aggregative games and illustrate our results on an electricity market game.
Regularization for Covariance Parameterization of Direct Data-Driven LQR Control
Zhao, Feiran; Chiuso, Alessandro; Dörfler, Florian (2025)
IEEE Control Systems Letters
As the benchmark of data-driven control methods, the linear quadratic regulator (LQR) problem has gained significant attention. A growing trend is direct LQR design, which finds the optimal LQR gain directly from raw data and bypassing system identification. To achieve this, our previous work develops a direct LQR formulation parameterized by sample covariance. In this letter, we propose a regularization method for the covariance-parameterized LQR. We show that the regularizer accounts for the uncertainty in both the steady-state covariance matrix corresponding to closed-loop stability, and the LQR cost function corresponding to averaged control performance. With a positive or negative coefficient, the regularizer can be interpreted as promoting either exploitation or exploration, which are well-known trade-offs in reinforcement learning. In simulations, we observe that our covariance-parameterized LQR with regularization can significantly outperform the certainty-equivalence LQR in terms of both the optimality gap and the robust stability.
How to Represent and Identify Affine Time-Invariant Systems?
Markovsky, Ivan; Eising, Jaap; Padoan, Alberto (2025)
IEEE Control Systems Letters
Affine systems are ubiquitous in modeling and emerge naturally from the linearization of nonlinear dynamics. Despite their relevance in applications, their identification remains largely ad hoc, relying on centering the data before applying linear identification methods. This heuristic approach assumes constant offset and can introduce bias. We develop a dedicated framework for affine system identification, deriving identifiability conditions and identification methods based on difference equation representations. Unlike the classical two-step approach, our method identifies the data-generating system under conditions verifiable from data and system complexity. For noisy data in the errors-in-variables setting, we recast the problem as a structured low-rank approximation, leveraging existing optimization techniques for efficient computation.

Publications 1 - 10 of 72

Journal: IEEE Control Systems Letters

Abbreviation

Publisher

Journal Volumes

ISSN

Description

Filters

Search Results