Journal of Modern Power Systems and Clean Energy

ISSN 2196-5625 CN 32-1884/TK

Two-timescale Volt/var Control Based on Reinforcement Learning with Hybrid Action Space for Distribution Networks
CSTR:
Author:
Affiliation:

1.School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China;2.School of Intelligent Manufacturing Engineering and Future Technologies, Fuyao University of Science and Technology, Fuzhou 350001, China;3.School of Automation and Electrical Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China

Clc Number:

Fund Project:

This work was supported in part by the National Science and Technology Major Project (No. 2022ZD0116900), the National Natural Science Foundation of China (No. 52277118), and the Natural Science Foundation of Tianjin (No. 22JCZDJC00660).

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    In volt/var control (VVC) for active distribution networks, it is essential to integrate traditional voltage regulation devices with modern smart photovoltaic inverters to prevent voltage violations. However, model-based multi-device VVC methods rely on accurate system models for decision-making, which can be challenging due to the extensive modeling workload. To tackle the complexities of multi-device cooperation in VVC, this paper proposes a two-timescale VVC method based on reinforcement learning with hybrid action space, termed the hybrid action representation twin delayed deep deterministic policy gradient (HAR-TD3) method. This method simultaneously manages traditional discrete voltage regulation devices, which operate on a slower timescale, and smart continuous voltage regulation devices, which function on a faster timescale. To enable effective collaboration between the different action spaces of these devices, we propose a variational auto-encoder based hybrid action reconstruction network. This network captures the interdependencies of hybrid actions by embedding both discrete and continuous actions into the latent representation space and subsequently decoding them for action reconstruction. The proposed method is validated on IEEE 33-bus, 69-bus, and 123-bus distribution networks. Numerical results indicate that the proposed method successfully coordinates discrete and continuous voltage regulation devices, achieving fewer voltage violations compared with state-of-the-art reinforcement learning methods.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:June 18,2024
  • Revised:October 08,2024
  • Adopted:
  • Online: February 13,2026
  • Published:
Article QR Code