Journal of Modern Power Systems and Clean Energy

ISSN 2196-5625 CN 32-1884/TK

Graph-based Safe Reinforcement Learning for Dynamic Optimal Power Flow with Hybrid Action Space Considering Time-varying Network Topologies
CSTR:
Author:
Affiliation:

1.School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China.;2.Department of Electrical and Electronic Engineering, School of Engineering, Cardiff University, Cardiff, CF24 3AA, UK;3.School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China;4.School of Electrical Engineering and Automation, Fuzhou University, Fuzhou 310108, China

Clc Number:

Fund Project:

This work was supported by the Tianjin Science and Technology Program (No. 22JCZDJC00820).

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    The proliferation of distributed energy resources and time-varying network topologies in active distribution networks presents unprecedented challenges for network operators. While reinforcement learning (RL) has shown promise in addressing network-constrained energy scheduling, it faces difficulties in managing the complexities of dynamic topologies and discrete-continuous hybrid action spaces. To address these challenges, a graph-based safe RL approach is proposed to learn dynamic optimal power flow under time-varying network topologies. This proposed approach leverages graph convolution operators to handle network topology changes, while safe RL with parameterized action ensures policy development. Specifically, the graph convolution operator abstracts key characteristics of the network topology, enabling effective power flow management in non-stationary environments. Besides that, a parameterized action constrained Markov decision process is employed to handle the hybrid action space and ensure compliance with physical network constraints, thereby accelerating the deployment of safe policy for hybrid action spaces. Numerical results demonstrate that the proposed approach efficiently navigates the discrete-continuous decision space while accounting for the constraints imposed by the dynamic nature of power flow in time-varying network topologies.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:November 05,2024
  • Revised:February 11,2025
  • Adopted:
  • Online: January 30,2026
  • Published:
Article QR Code