Florian Dörfler

Last Name

Dörfler

First Name

Florian

ORCID

0000-0002-9649-5305

Organisational unit

09478 - Dörfler, Florian / Dörfler, Florian

Show all metadata

Search Results

Publications 1 - 10 of 228

Online Feedback Optimization over Networks: A Distributed Model-Free Approach
Wang, Wenbin; He, Zhiyu; Belgioioso, Giuseppe; et al. (2024)
2024 IEEE 63rd Conference on Decision and Control (CDC)
Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be easily accessible in practice. Motivated by these challenges, we develop a fully distributed model-free OFO controller, featuring consensus-based tracking of the global objective value and local iterative (projected) updates that use stochastic gradient estimates. We characterize how the closed-loop performance depends on the size of the network, the number of iterations, and the level of accuracy of consensus. Numerical simulations on a voltage control problem in a direct current power grid corroborate the theoretical findings.
Input–Output Performance of Linear–Quadratic Saddle-Point Algorithms With Application to Distributed Resource Allocation Problems
Simpson-Porco, John W.; Poolla, Bala Kameshwar; Monshizadeh, Nima; et al. (2020)
IEEE Transactions on Automatic Control
Dynamic Programming in Probability Spaces via Optimal Transport
Terpin, Antonio; Lanzetti, Nicolas; Dörfler, Florian (2024)
SIAM Journal on Control and Optimization
We study discrete-time finite-horizon optimal control problems in probability spaces, whereby the state of the system is a probability measure. We show that, in many instances, the solution of dynamic programming in probability spaces results from two ingredients: (i) the solution of dynamic programming in the “ground space” (i.e., the space on which the probability measures live) and (ii) the solution of an optimal transport problem. From a multi-agent control perspective, a separation principle holds: “low-level control of the agents of the fleet” (how does one reach the destination?) and “fleet-level control” (who goes where?) are decoupled.
Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions
Terpin, Antonio; Lanzetti, Nicolas; Yardim, Batuhan; et al. (2022)
arXiv
Policy Optimization (PO) algorithms have been proven particularly suited to handle the high-dimensionality of real-world continuous control tasks. In this context, Trust Region Policy Optimization methods represent a popular approach to stabilize the policy updates. These usually rely on the Kullback-Leibler (KL) divergence to limit the change in the policy. The Wasserstein distance represents a natural alternative, in place of the KL divergence, to define trust regions or to regularize the objective function. However, state-of-the-art works either resort to its approximations or do not provide an algorithm for continuous state-action spaces, reducing the applicability of the method. In this paper, we explore optimal transport discrepancies (which include the Wasserstein distance) to define trust regions, and we propose a novel algorithm - Optimal Transport Trust Region Policy Optimization (OT-TRPO) - for continuous state-action spaces. We circumvent the infinite-dimensional optimization problem for PO by providing a one-dimensional dual reformulation for which strong duality holds. We then analytically derive the optimal policy update given the solution of the dual problem. This way, we bypass the computation of optimal transport costs and of optimal transport maps, which we implicitly characterize by solving the dual formulation. Finally, we provide an experimental evaluation of our approach across various control tasks. Our results show that optimal transport discrepancies can offer an advantage over state-of-the-art approaches.
Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions
Terpin, Antonio; Lanzetti, Nicolas; Yardim, Batuhan; et al. (2022)
Advances in Neural Information Processing Systems 35
Policy Optimization (PO) algorithms have been proven particularly suited to handle the high-dimensionality of real-world continuous control tasks. In this context, Trust Region Policy Optimization methods represent a popular approach to stabilize the policy updates. These usually rely on the Kullback-Leibler (KL) divergence to limit the change in the policy. The Wasserstein distance represents a natural alternative, in place of the KL divergence, to define trust regions or to regularize the objective function. However, state-of-the-art works either resort to its approximations or do not provide an algorithm for continuous state-action spaces, reducing the applicability of the method.In this paper, we explore optimal transport discrepancies (which include the Wasserstein distance) to define trust regions, and we propose a novel algorithm - Optimal Transport Trust Region Policy Optimization (OT-TRPO) - for continuous state-action spaces. We circumvent the infinite-dimensional optimization problem for PO by providing a one-dimensional dual reformulation for which strong duality holds.We then analytically derive the optimal policy update given the solution of the dual problem. This way, we bypass the computation of optimal transport costs and of optimal transport maps, which we implicitly characterize by solving the dual formulation.Finally, we provide an experimental evaluation of our approach across various control tasks. Our results show that optimal transport discrepancies can offer an advantage over state-of-the-art approaches.
Optimizing Social Network Interventions via Hypergradient-Based Recommender System Design
Kühne, Marino; Grontas, Panagiotis D.; De Pasquale, Giulia; et al. (2025)
Proceedings of Machine Learning Research ~ Proceedings of the 42nd International Conference on Machine Learning
Although social networks have expanded the range of ideas and information accessible to users, they are also criticized for amplifying the polarization of user opinions. Given the inherent complexity of these phenomena, existing approaches to counteract these effects typically rely on handcrafted algorithms and heuristics. We propose an elegant solution: we act on the network weights that model user interactions on social networks (e.g., ranking of users’ shared content in feeds), to optimize a performance metric (e.g., minimize polarization), while users’ opinions follow the classical Friedkin-Johnsen model. Our formulation gives rise to a challenging, large-scale optimization problem with non-convex constraints, for which we develop a gradient-based algorithm. Our scheme is simple, scalable, and versatile, as it can readily integrate different, potentially non-convex, objectives. We demonstrate its merit by: (i) rapidly solving complex social network intervention problems with 4.8 million variables based on the Reddit, LiveJournal, and DBLP datasets; (ii) outperforming competing approaches in terms of both computation time and disagreement reduction.
Gaussian behaviors: representations and data-driven control
Sasfi, András; Markovsky, Ivan; Padoan, Alberto; et al. (2025)
We propose a modeling framework for stochastic systems, termed Gaussian behaviors, that describes finite-length trajectories of a system as a Gaussian process. The proposed model naturally quantifies the uncertainty in the trajectories, yet it is simple enough to allow for tractable formulations. We relate the proposed model to existing descriptions of dynamical systems including deterministic and stochastic behaviors, and linear time-invariant (LTI) state-space models with Gaussian noise. Gaussian behaviors can be estimated directly from observed data as the empirical sample covariance. The distribution of future outputs conditioned on inputs and past outputs provides a predictive model that can be incorporated in predictive control frameworks. We show that subspace predictive control is a certainty-equivalence control formulation with the estimated Gaussian behavior. Furthermore, the regularized data-enabled predictive control (DeePC) method is shown to be a distributionally optimistic formulation that optimistically accounts for uncertainty in the Gaussian behavior. To mitigate the excessive optimism of DeePC, we propose a novel distributionally robust control formulation, and provide a convex reformulation allowing for efficient implementation.
Optimality of Linear Policies for Distributionally Robust Linear Quadratic Gaussian Regulator with Stationary Distributions
Lanzetti, Nicolas; Terpin, Antonio; Dörfler, Florian (2024)
arXiv
We prove that linear policies remain optimal for solving the Linear Quadratic Gaussian regulation problem in face of worst-case process and measurement noise distributions, when these are independent, stationary, and known to be within a radius (in the Wasserstein sense) to some reference Gaussian noise distributions. When the reference noise distributions are not Gaussian, we provide a closed-form solution for the worst-case distributions. Our main result suggests a computational complexity that scales only with the dimensionality of the system, and provides a less conservative alternative to recent work in distributionally robust control.
Variational Analysis in the Wasserstein Space
Lanzetti, Nicolas; Terpin, Antonio; Dörfler, Florian (2024)
arXiv
We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which however involve projections or rely on convexity-type assumptions. We believe instead that these problems call for a comprehensive methodological framework for calculus in probability spaces. In this work, we combine ideas from optimal transport, variational analysis, and Wasserstein gradient flows to equip the Wasserstein space (i.e., the space of probability measures endowed with the Wasserstein distance) with a variational structure, both by combining and extending existing results and introducing novel tools. Our theoretical analysis culminates in very general necessary optimality conditions for optimality. Notably, our conditions (i) resemble the rationales of Euclidean spaces, such as the Karush-Kuhn-Tucker and Lagrange conditions, (ii) are intuitive, informative, and easy to study, and (iii) yield closed-form solutions or can be used to design computationally attractive algorithms. We believe this framework lays the foundation for new algorithmic and theoretical advancements in the study of optimization problems in probability spaces, which we exemplify with numerous case studies and applications to machine learning, drug discovery, and distributionally robust optimization.
Learning diffusion at lightspeed
Terpin, Antonio; Lanzetti, Nicolas; Gadea, Martín; et al. (2024)
Advances in Neural Information Processing Systems 37
Diffusion regulates numerous natural processes and the dynamics of many successful generative models. Existing models to learn the diffusion terms from observational data rely on complex bilevel optimization problems and model only the drift of the system. We propose a new simple model, JKOnet*, which bypasses the complexity of existing architectures while presenting significantly enhanced representational capabilities: JKOnet* recovers the potential, interaction, and internal energy components of the underlying diffusion process. JKOnet* minimizes a simple quadratic loss and outperforms other baselines in terms of sample efficiency, computational complexity, and accuracy. Additionally, JKOnet* provides a closed-form optimal solution for linearly parametrized functionals, and, when applied to predict the evolution of cellular processes from real-world data, it achieves state-of-the-art accuracy at a fraction of the computational cost of all existing methods. Our methodology is based on the interpretation of diffusion processes as energy-minimizing trajectories in the probability space via the so-called JKO scheme, which we study via its first-order optimality conditions.

Publications 1 - 10 of 228

Florian Dörfler

Last Name

First Name

ORCID

Organisational unit

Filters

Search Results