Safe Reinforcement Learning for Strategic Bidding of Virtual Power Plants in Day-Ahead Markets
OPEN ACCESS
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
This paper presents a novel safe reinforcement learning algorithm for strategic bidding of Virtual Power Plants (VPPs) in day-ahead electricity markets. The proposed algorithm utilizes the Deep Deterministic Policy Gradient (DDPG) method to learn competitive bidding policies without requiring an accurate market model. Furthermore, to account for the complex internal physical constraints of VPPs, we introduce two enhancements to the DDPG method. Firstly, a projection-based safety shield that restricts the agent’s actions to the feasible space defined by the non-linear power flow equations and operating constraints of distributed energy resources is derived. Secondly, a penalty for the shield activation in the reward function that incentivizes the agent to learn a safer policy is introduced. A case study based on the IEEE 13-bus network demonstrates the effectiveness of the proposed approach in enabling the agent to learn a highly competitive, safe strategic policy.
Permanent link
Publication status
published
Editor
Book title
2023 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm)
Journal / series
Volume
Pages / Article No.
10333971
Publisher
IEEE
Event
14th IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm 2023)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Virtual power plants; Strategic bidding; Electricity markets; Safe reinforcement learning
Organisational unit
09481 - Hug, Gabriela / Hug, Gabriela
Notes
Funding
180545 - NCCR Automation (phase I) (SNF)