Abstract:Grid-following voltage source converter (GFL-VSC) and grid-forming voltage source converter (GFM-VSC) have different dynamic characteristics for active power-frequency and reactive power-voltage supports of the power grid. This paper aims to clarify and recognize the difference between grid-following (GFL) and grid-forming (GFM) frequency-voltage support more intuitively and clearly. Firstly, the phasor model considering circuit constraints is established based on the port circuit equations of the converter. It is revealed that the voltage and active power linearly correspond to the horizontal and vertical axes in the phasor space referenced to the grid voltage phasor. Secondly, based on topological homology, GFL and GFM controls are transformed and mapped into different trajectories. The topological similarity of the characteristic curves for GFL and GFM controls is the essential cause of their uniformity. Based on the above model, it is indicated that GFL-VSC and GFM-VSC possess uniformity with regard to active power response, type of coupling, and phasor trajectory. They differ in synchronization, power coupling mechanisms, dynamics, and active power-voltage operation domain in the quasi-steady state. Case studies are undertaken on GFL-VSC and GFM-VSC integrated into a four-machine two-area system. Simulation results verify that the dynamic uniformity and difference of GFL-VSC and GFM-VSC are intuitively and comprehensively revealed.

Abstract:This study investigates a safe reinforcement learning algorithm for grid-forming (GFM) inverter based frequency regulation. To guarantee the stability of the inverter-based resource (IBR) system under the learned control policy, a model-based reinforcement learning (MBRL) algorithm is combined with Lyapunov approach, which determines the safe region of states and actions. To obtain near optimal control policy, the control performance is safely improved by approximate dynamic programming (ADP) using data sampled from the region of attraction (ROA). Moreover, to enhance the control robustness against parameter uncertainty in the inverter, a Gaussian process (GP) model is adopted by the proposed algorithm to effectively learn system dynamics from measurements. Numerical simulations validate the effectiveness of the proposed algorithm.

Abstract:The high proportion of renewable energy integration and the dynamic changes in grid topology necessitate the enhancement of voltage/var control (VVC) to manage voltage fluctuations more rapidly. Traditional model-based control algorithms are becoming increasingly incompetent for VVC due to their high model dependence and slow online computation speed. To alleviate these issues, this paper introduces a graph attention network (GAT) based deep reinforcement learning for VVC of topologically variable power system. Firstly, combining the physical information of the actual power grid, a physics-informed GAT is proposed and embedded into the proximal policy optimization (PPO) algorithm. The GAT-PPO algorithm can capture topological and spatial correlations among the node features to tackle topology changes. To address the slow training, the ReliefF-S algorithm identifies critical state variables, significantly reducing the dimensionality of state space. Then, the training samples retained in the experience buffer are designed to mitigate the sparse reward issue. Finally, the validation on the modified IEEE 39-bus system and an actual power grid demonstrates superior performance of the proposed algorithm compared with state-of-the-art algorithms, including PPO algorithm and twin delayed deep deterministic policy gradient (TD3) algorithm. The proposed algorithm exhibits enhanced convergence during training, faster solution speed, and improved VVC performance, even in scenarios involving grid topology changes and increased renewable energy integration. Meanwhile, in the adopted cases, the network loss is reduced by 6.9%, 10.8%, and 7.7%, respectively, demonstrating favorable economic outcomes.

Abstract:Joint chance constraints (JCCs) can ensure the consistency and correlation of stochastic variables when participating in decision-making. Sample average approximation (SAA) is the most popular method for solving JCCs in unit commitment (UC) problems. However, the typical SAA requires large Monte Carlo (MC) samples to ensure the solution accuracy, which results in large-scale mixed-integer programming (MIP) problems. To address this problem, this paper presents the partial sample average approximation (PSAA) to deal with JCCs in UC problems in multi-area power systems with wind power. PSAA partitions the stochastic variables and historical dataset, and the historical dataset is then partitioned into non-sampled and sampled sets. When approximating the expectation of stochastic variables, PSAA replaces the big-M formulation with the cumulative distribution function of the non-sampled set, thus preventing binary variables from being introduced. Finally, PSAA can transform the chance constraints to deterministic constraints with only continuous variables, avoiding the large-scale MIP problem caused by SAA. Simulation results demonstrate that PSAA has significant advantages in solution accuracy and efficiency compared with other existing methods including traditional SAA, SAA with improved big-M, SAA with Latin hypercube sampling (LHS), and the multi-stage robust optimization methods.

Abstract:This paper establishes a two-layer data-driven robust scheduling method to deal with the significant computational complexity and uncertainties in scheduling industrial heat loads. First, a two-layer deterministic scheduling model is proposed to address the computational burden of utilizing flexibility from a large number of bitumen tanks (BTs). The key feature of this model is the capability to reduce the number of control variables through analyzing and modeling the clustered temperature transfer of BTs. Second, to tackle the uncertainties in the scheduling problem, historical data regarding BTs are collected and analyzed, and a data-driven piecewise linear Kernel-based support vector clustering technique is employed to construct the uncertainty set with convex boundaries and adjustable conservatism, based on which robust optimization can be conducted. The case results indicate that the proposed method enables the utilization of flexibility in BTs, improving the level of onsite photovoltaic consumption and reducing the aggregated load fluctuation.

Abstract:The integration of distributed energy resources (DERs) has escalated the challenge of voltage magnitude regulation in distribution networks. Model-based approaches, which rely on complex sequential mathematical formulations, cannot meet the real-time demand. Deep reinforcement learning (DRL) offers an alternative by utilizing offline training with distribution network simulators and then executing online without computation. However, DRL algorithms fail to enforce voltage magnitude constraints during training and testing, potentially leading to serious operational violations. To tackle these challenges, we introduce a novel safe-guaranteed reinforcement learning algorithm, the DistFlow safe reinforcement learning (DF-SRL), designed specifically for real-time voltage magnitude regulation in distribution networks. The DF-SRL algorithm incorporates a DistFlow linearization to construct an expert-knowledge-based safety layer. Subsequently, the DF-SRL algorithm overlays this safety layer on top of the agent policy, recalibrating unsafe actions to safe domains through a quadratic programming formulation. Simulation results show the DF-SRL algorithm consistently ensures voltage magnitude constraints during training and real-time operation (test) phases, achieving faster convergence and higher performance, which differentiates it apart from (safe) DRL benchmark algorithms.

Abstract:The increasing penetration of renewable energy resources and reduced system inertia pose risks to frequency security of power systems, necessitating the development of fast frequency regulation (FFR) methods using flexible resources. However, developing effective FFR policies is challenging because different power system operating conditions require distinct regulation logics. Traditional fixed-coefficient linear droop-based control methods are suboptimal for managing the diverse conditions encountered. This paper proposes a dynamic nonlinear P-f droop-based FFR method using a newly established meta-reinforcement learning (meta-RL) approach to enhance control adaptability while ensuring grid stability. First, we model the optimal FFR problem under various operating conditions as a set of Markov decision processes and accordingly formulate the frequency stability-constrained meta-RL problem. To address this, we then construct a novel hierarchical neural network (HNN) structure that incorporates a theoretical frequency stability guarantee, thereby converting the constrained meta-RL problem into a more tractable form. Finally, we propose a two-stage algorithm that leverages the inherent characteristics of the problem, achieving enhanced optimality in solving the HNN-based meta-RL problem. Simulations validate that the proposed FFR method shows superior adaptability across different operating conditions, and achieves better trade-offs between regulation performance and cost than benchmarks.

Abstract:As the proportion of renewable energy sources continues to increase, the local damping contributions of sources in power system decrease, posing a challenge to the power system stability. Therefore, online tracking of the damping contributions of each source is crucial for the prevention of low-frequency oscillations. This paper proposes an online tracking method of local damping under ambient data. The proposed method is based on dissipation energy spectrum analysis (DESA) and the energy dissipation factor (EDF). First, the feasibility of using frequency-domain analysis for the dissipation energy of generator is analyzed. The frequency spectral function of dissipation energy of generator is then derived by integrating with Parseval’s theorem, and the EDF is defined. Second, the generator energy dissipation factor (GEDF) for the dominant oscillation mode frequency is established. The modal information of the dominant oscillation in the power system is obtained through DESA. The relationship between the frequency spectral function and eigenvalues is also established. Finally, an online tracking method of local damping is proposed based on DESA and GEDF. The effectiveness of the proposed method is validated through simulations on a four-machine 11-bus power system and an actual power system in Northwest China.

Abstract:This paper studies the problem of multi-stage robust unit commitment with discrete load shedding. In the day-ahead phase, the on-off status of thermal units is scheduled. During each period of real-time dispatch, the output of thermal units and the action of load shedding are determined, and the discrete choice of load shedding corresponds to the practice of tripping substation outlets. The entire decision-making process is formulated as a multi-stage adaptive robust optimization problem with mixed-integer recourse, whose solution takes three steps. First, we propose and apply partially affine policy, which is optimized ahead of the day and restricts intertemporal dispatch variables as affine functions of previous uncertainty realizations, leaving remaining continuous and binary dispatch variables to be optimized in real time. Second, we demonstrate that the resulting model with partially affine policy can be reformulated as a two-stage robust optimization problem with mixed-integer recourse. Third, we modify the standard nested column-and-constraint generation algorithm to accelerate the inner loops by warm start. The modified algorithm solves the two-stage problem more efficiently. Case studies on the IEEE 118-bus system verify that the proposed partially affine policy outperforms conventional affine policy in terms of optimality and robustness; the modified nested column-and-constraint generation algorithm significantly reduces the total computation time; and the proposed method balances well optimality and efficiency compared with state-of-the-art methods.

Abstract:The gradual replacement of gasoline vehicles with electric vehicles (EVs) and hydrogen fuel cell vehicles (HFCVs) in recent years has provided a growing incentive for the collaborative optimization of power distribution network (PDN), urban transportation network (UTN), and hydrogen distribution network (HDN). However, an appropriate collaborative optimization framework that addresses the prevalent privacy concerns has yet to be developed, and a sufficient pool of system operators that can competently operate all three networks has yet to be obtained. This study proposes a differentiated taxation-subsidy mechanism for UTNs, utilizing congestion tolls and subsidies to guide the independent traffic flow of EVs and HFCVs. An integrated optimization model for this power-hydrogen-transportation network is established by treating these vehicles and the electrolysis equipment as coupling bridges. We then develop a learning-aided decoupling approach to determine the values of the coupling variables acting among the three networks to ensure the economic feasibility of collaborative optimization. This approach effectively decouples the network, allowing it to operate and be optimized independently. The results for a numerical simulation of a coupled system composed of a IEEE 33-node power network, 13-node Nguyen-Dupuis transportation network, and 20-node HDN demonstrate that the proposed learning-aided approach provides nearly equivalent dispatching results as those derived from direct solution of the physical models of the coupled system, while significantly improving the computational efficiency.

Abstract:The variable and unpredictable nature of renewable energy generation (REG) presents challenges to its large-scale integration and the efficient and economic operation of the electricity network, particularly at the distribution level. In this paper, an operational coordination optimization method is proposed for the electricity and natural gas networks, aiming to overcome the identified negative impacts. The method involves the implementation of bi-directional energy flows through power-to-gas units and gas-fired power plants. A detailed model of the three-phase power distribution system up to each phase is employed to improve the representation of multi-energy systems to consider real-world end-user consumption. This method allows for the full consideration of unbalanced operational scenarios. Meanwhile, the natural gas network is modelled and analyzed with steady-state gas flows and the dynamics of the line pack in pipelines. The sequential symmetrical second-order cone programming (SS-SOCP) method is employed to facilitate the simultaneous analysis of three-phase imbalance and line pack while accelerating the solution process. The efficacy of the operational coordination optimization method is demonstrated in case studies comprising a modified IEEE 123-node power distribution system with a 20-node natural gas network. The studies show that the operational coordination optimization method can simultaneously minimize the total operational cost, the curtailment of installed REG, the voltage imbalance of three-phase power system, and the overall carbon emissions.

Abstract:In the context of large-scale photovoltaic integration, flexibility scheduling is essential to ensure the secure and efficient operation of distribution networks (DNs). Recently, deep reinforcement learning (DRL) has been widely applied to scheduling problems. However, most methods neglect the vulnerability of DRL to state adversarial attacks such as load redistribution attacks, significantly undermining its security and reliability. To this end, a flexibility scheduling method is proposed based on robust graph DRL (RoGDRL). A flexibility gain improvement model considering temperature-dependent resistance is first proposed, which considers weather factors as additional variables to enhance the precision of flexibility analysis. Based on this, a state-adversarial two-player zero-sum Markov game (SA-TZMG) model is proposed, which converts the robust DRL scheduling problem into a Nash equilibrium problem. The proposed SA-TZMG model considers the physical constraints of state attacks that guarantee the maximal flexibility gain for the defender when confronted with the most sophisticated and stealthy attacker. A two-stage RoGDRL algorithm is proposed, which introduces the graph sample and aggregate (GraphSAGE) driven soft actor-critic to capture the complex feature about the neighbors of nodes and their properties via inductive learning, thereby solving the Nash equilibrium policies more efficiently. Simulations based on the modified IEEE 123-bus system demonstrates the efficacy of the proposed method.

Abstract:Peer-to-peer (P2P) energy trading in active distribution networks (ADNs) plays a pivotal role in promoting the efficient consumption of renewable energy sources. However, it is challenging to effectively coordinate the power dispatch of ADNs and P2P energy trading while preserving the privacy of different physical interests. Hence, this paper proposes a soft actor-critic algorithm incorporating distributed trading control (SAC-DTC) to tackle the optimal power dispatch of ADNs and the P2P energy trading considering privacy preservation among prosumers. First, the soft actor-critic (SAC) algorithm is used to optimize the control strategy of device in ADNs to minimize the operation cost, and the primary environmental information of the ADN at this point is published to prosumers. Then, a distributed generalized fast dual ascent method is used to iterate the trading process of prosumers and maximize their revenues. Subsequently, the results of trading are encrypted based on the differential privacy technique and returned to the ADN. Finally, the social welfare value consisting of ADN operation cost and P2P market revenue is utilized as a reward value to update network parameters and control strategies of the deep reinforcement learning. Simulation results show that the proposed SAC-DTC algorithm reduces the ADN operation cost, boosts the P2P market revenue, maximizes the social welfare, and exhibits high computational accuracy, demonstrating its practical application to the operation of power systems and power markets.

Abstract:The flexible interconnection of microgrids (MGs) adopting back-to-back converters (BTBCs) has emerged as a new development trend in the field of MGs. This approach enables larger-scale integration and higher utilization of distributed renewable energy sources (RESs). However,their stability characteristics are very different from single MG due to the control characteristics of flexible interconnection. Meanwhile, the uncertainty and stochastic dependence structures of RESs and loads create challenges for stability analysis and cooperative control. In this paper, a probabilistic small-signal stability assessment and cooperative control framework is proposed for interconnected MGs via BTBCs. First, a cooperative control architecture for MGs is constructed. Then, a small-signal model of interconnected MGs via BTBCs containing primary control and secondary control is developed. This model facilitates the analysis of the impacts of BTBCs and various control strategies on the system stability. Subsequently, Copula functions and polynomial chaos expansion (PCE) are combined to achieve the probabilistic small-signal stability assessment. On this basis, the parameters of the cooperative control are optimized, enhancing the robustness of interconnected MGs via BTBCs. Finally, a case of interconnected MGs via BTBCs are built in MATLAB/Simulink to verify the accuracy and effectiveness of the proposed framework.

Abstract:The optimal dispatch of energy storage systems (ESSs) in distribution networks poses significant challenges, primarily due to uncertainties of dynamic pricing, fluctuating demand, and the variability inherent in renewable energy sources. By exploiting the generalization capabilities of deep neural networks (DNNs), the deep reinforcement learning (DRL) algorithms can learn good-quality control models that adapt to the stochastic nature of distribution networks. Nevertheless, the practical deployment of DRL algorithms is often hampered by their limited capacity for satisfying operational constraints in real time, which is a crucial requirement for ensuring the reliability and feasibility of control actions during online operations. This paper introduces an innovative framework, named mixed-integer programming based deep reinforcement learning (MIP-DRL), to overcome these limitations. The proposed MIP-DRL framework can rigorously enforce operational constraints for the optimal dispatch of ESSs during the online execution. This framework involves training a Q-function with DNNs, which is subsequently represented in a mixed-integer programming (MIP) formulation. This unique combination allows for the seamless integration of operational constraints into the decision-making process. The effectiveness of the proposed MIP-DRL framework is validated through numerical simulations, demonstrating its superior capability to enforce all operational constraints and achieve high-quality dispatch decisions and showing its advantage over existing DRL algorithms.

Abstract:The distinctive fault characteristics of battery energy storage stations (BESSs) significantly affect the reliability of conventional protection methods for transmission lines. In this paper, the three-dimensional (3D) data scattergrams are constructed using current data from both sides of the transmission line and their sum. Following a comprehensive analysis of the varying characteristics of 3D data scattergrams under different conditions, a 3D data scattergram image classification based protection method is developed. The depth-wise separable convolution is used to ensure a lightweight convolutional neural network (CNN) structure without compromising performance. In addition, a Bayesian hyperparameter optimization algorithm is used to achieve a hyperparametric search to simplify the training process. Compared with artificial neural networks and CNNs, the depth-wise separable convolution based CNN (DPCNN) achieves a higher recognition accuracy. The 3D data scattergram image classification based protection method using DPCNN can accurately separate internal faults from other disturbances and identify fault phases under different operating states and fault conditions. The proposed protection method also shows first-class tolerability against current transformer (CT) saturation and CT measurement errors.

Abstract:With the development of the carbon markets (CMs) and electricity markets (EMs), discrepancies in prices between the two markets and between two time periods offer profit opportunities for generation companies (GenCos). Motivated by the carbon option and Black-Scholes (B-S) model, GenCos are given the right but not the obligation to trade carbon emission allowances (CEAs) and use instruments to hedge against price risks. To model the strategic behaviors of GenCos that capitalize on these cross-market and cross-time opportunities, a multi-market trading strategy that incorporates option-jointed daily trading and reinforcement learning-jointed weekly continuous trading are modeled. The daily trading is built with a bi-level structure, where a profit-oriented bidding model that jointly considers both the optimal CEA holding shares and the best bidding curves is developed at the upper level. At the lower level, in addition to market clearing models of the day-ahead EM and auction-based CM, a B-S model that considers carbon trading asynchronism and option pricing is constructed. Then, by expanding the daily trading, the weekly continuous trading is modeled and solved using reinforcement learning. Binary expansion and strike-to-spot price ratio are utilized to address the nonlinearity. Finally, case studies on an IEEE 30-bus system are conducted to validate the effectiveness of the proposed trading strategy. Results show that the proposed trading strategy can increase GenCo profits by influencing market prices and leveraging carbon options.

Abstract:The spot flexibility markets are before the real-time energy exchange, allowing demand-side management to reduce energy consumption during peak periods. In these markets, demand aggregators must quickly choose the customers ’reduction bids that fulfill grid requirements. This clearing procedure is challenging due to the computational complexity of selecting the optimal bids. Therefore, developing a clearing mechanism that avoids searching the entire flexibility bid space while respecting grid constraints is essential for the smooth operation of the spot flexibility market. This paper presents a clearing mechanism with reduced computational complexity of the winner determination problem in spot flexibility market for demand aggregators carrying out reductions in energy consumption. The proposed approach transforms customers’flexibility bids into a reward-based function. Afterward, the gradient-based optimization solves the bid selection problem. This approach helps demand aggregators achieve satisfactory energy reductions within an appropriate delay for spot flexibility markets. A comparative study presents the effectiveness of the proposed approach against commonly used approaches: hybrid particle swarm optimization genetic algorithm and combinatorial search.

Abstract:With the rapid integration of renewable energy, wide-band oscillations caused by interactions between power electronic equipment and grids have emerged as one of the most critical stability issues. Existing methods are usually studied for local power systems with around one hundred nodes. However, for a large-scale power system with tens of thousands of nodes, the dimension of transfer function matrix or the order of characteristic equation is much higher. In this case, the existing methods such as eigenvalue analysis method and impedance-based method have difficulty in computation and are thus hard to utilize in practice. To fill this gap, this paper proposes a novel method named the smallest eigenvalues based logarithmic derivative (SELD) method. It obtains the dominant oscillation modes by the logarithmic derivative of the k-smallest eigenvalue curves of the sparse extended nodal admittance matrix (NAM). An oscillatory stability analysis tool is further developed based on this method. The effectiveness of the method and the tool is validated through a local power system as well as a large-scale power system.

Abstract:Realistic uncertainties of renewable energies and loads may possess complicated probability distributions and correlations, which are difficult to be characterized by standard probability density functions and hence challenge existing uncertainty propagation analysis (UPA) methods. Also, nonintrusive spectral representation (SR)-based UPA methods can only estimate system responses at each time point separately, which is time-consuming for analyzing power system dynamics. Thus, this paper proposes a generic multi-output SR (GMSR) method to effectively tackle the above limitations by developing the generic correlation transformation and multi-output structure. The effectiveness and superiority of GMSR in efficiency and accuracy are demonstrated by comparing it with existing SR methods.

Abstract:Under-frequency load shedding (UFLS) serves as the very last resort for preventing total blackouts and cascading events. Fluctuating operating conditions and weak resilience of the future grid require UFLS adapt to various operating conditions and non-envisioned faults. This paper develops a novel data-enabled Koopman-based load shedding (KLS) to achieve the optimal one-shot load shedding for power system frequency safety. The KLS yields a network that facilitates a coordinate transformation from the delay-embedded space to a new space, wherein the dynamics can be expressed in a linear manner. The network is specifically tailored to effectively track parameter variations in the dynamic model of the power system. Linear dynamics support the development of a real-time decided load shedding strategy, while parameter tracking enables the adaptability of the KLS to non-envisioned operating conditions and faults. To address approximation inaccuracies and the discrete nature of load shedding, a safety margin tuning scheme is integrated into the KLS framework, ensuring that the system frequency trajectory remains within the safety range. Simulation results show the adaptability, prediction capability, and control effect of the proposed KLS.

Abstract:The increasing penetration of renewable energy resources degrades the frequency stability of power systems. The present work addresses this issue by proposing a look-ahead dispatch model of power systems based on a linear alternating current optimal power flow framework with nonlinear frequency constraints. Meanwhile, the poor efficiency for solving this formulation is addressed by introducing a physics-informed neural network (PINN) to predict key frequency-control parameter values accurately. The PINN ensures that the learned results are applicable to the original physical frequency dynamics model, and applying the predicted parameter values enables the resulting dispatch model to be solved quickly and efficiently using readily available commercial solvers. The feasibility and advantages of the proposed model are demonstrated by the results of numerical computations applied to a modified IEEE 118-bus test system.

Abstract:The oscillatory stability analysis of multi-converter-fed systems (MCFSs) with modest computational resources needs a precise parametric reduced-order impedance model (PROIM). However, the traditional Krylov subspace based parametric model order reduction (KS-PMOR) method has difficulty in building precise PROIM for MCFSs. This is because the factors related to the errors of PROIM are complicated and coupled. To fill this gap, the factors associated with the accuracy of the KS-PMOR method are estimated by defining three indicators: the convergence error, cumulative error, and rank of projection matrix. Using the three indicators, a frequency-domain adaptive parametric model order reduction (FDA-PMOR) method is developed to form the precise PROIM of MCFSs for the accurate and fast oscillatory stability analysis. The accuracy of the obtained PROIM using the proposed FDA-PMOR method and its efficiency in actual oscillatory stability analysis are validated by three MCFSs with different scales, i.e., a small-scale MCFS with four paralleled converter-based renewable energy generators (CREGs), a real-time simulation-based MCFS with eighteen paralleled CREGs, and a larger MCFS with ninety paralleled CREGs.

Abstract:Traditional protection methods are not suitable for hybrid (cable and overhead) transmission lines in voltage source converter based high-voltage direct current (VSC-HVDC) systems. Accordingly, this paper presents the robust fault detection, classification, and location based on the empirical wavelet transform-Teager energy operator (EWT-TEO) and artificial neural network (ANN) for hybrid transmission lines in VSC-HVDC systems. The operational scheme of the proposed protection method consists of two loops ① an EWT-TEO based feature extraction loop, ② and an ANN-based fault detection, classification, and location loop. Under the proposed protection method, the voltage and current signals are decomposed into several sub-passbands with low and high frequencies using the empirical wavelet transform (EWT) method. The energy content extracted by the EWT is fed into the ANN for fault detection, classification, and location. Various fault cases, including the high-impedance fault (HIF) as well as noises, are performed to train the ANN with two hidden layers. The test system and signal decomposition are conducted by PSCAD/EMTDC and MATLAB, respectively. The performance of the proposed protection method is compared with that of the traditional non-pilot traveling wave (TW) based protection method. The results confirm the high accuracy of the proposed protection method for hybrid transmission lines in VSC-HVDC systems, where a mean percentage error of approximately 0.1% is achieved.

Abstract:Building integrated energy systems (BIESs) are pivotal for enhancing energy efficiency by accounting for a significant proportion of global energy consumption. Two key barriers that reduce the BIES operational efficiency mainly lie in the renewable generation uncertainty and operational non-convexity of combined heat and power (CHP) units. To this end, this paper proposes a soft actor-critic (SAC) algorithm to solve the scheduling problem of BIES, which overcomes the model non-convexity and shows advantages in robustness and generalization. This paper also adopts a temporal fusion transformer (TFT) to enhance the optimal solution for the SAC algorithm by forecasting the renewable generation and energy demand. The TFT can effectively capture the complex temporal patterns and dependencies that span multiple steps. Furthermore, its forecasting results are interpretable due to the employment of a self-attention layer so as to assist in more trustworthy decision-making in the SAC algorithm. The proposed hybrid data-driven approach integrating TFT and SAC algorithm, i.e., TFT-SAC approach, is trained and tested on a real-world dataset to validate its superior performance in reducing the energy cost and computational time compared with the benchmark approaches. The generalization performance for the scheduling policy, as well as the sensitivity analysis, are examined in the case studies.

Abstract:A novel planning tool for optimizing the placement of electric springs (ESs) in unbalanced distribution networks is introduced in this study. The total voltage deviation is used as the optimization criterion and is calculated when the ESs operate at their maximum reactive power either in the inductive or capacitive modes. The power rating of the ES is adjusted on the basis of the available active power at the bus. And in the optimization problem, it is expressed as the power ratio of the noncritical load (NCL) and critical load (CL). The implemented ES model is flexible, which can be used on any bus and any phase. The model determines the output voltage from the parameters and operating conditions at the point of common coupling (PCC). These conditions are integrated using the backward/forward sweep method (BFSM) and are updated during power flow calculations. The problem is described as a mixed-integer nonlinear problem and solved efficiently using an improved BFSM-based genetic algorithm, which computes power flow and ES placement simultaneously. The effectiveness of this method is evaluated through testing in IEEE 13-bus and 34-bus systems.

Abstract:This paper proposes a novel parallel hybrid deep reinforcement learning (DRL) approach to address the real-time energy management problem for microgrid (MG). As the proposed approach can directly approximate a discrete-continuous hybrid policy, it does not require the discretization of continuous actions like regular DRL approaches, which avoids accuracy degradation and the curse of dimensionality. In addition, a novel experience-sharing-based parallel technique is further developed for the proposed approach to accelerate the training speed and enhance the training robustness. Finally, a safety projection technique is introduced and incorporated into the proposed approach to improve the decision feasibility. Comparative numerical simulations with several existing MG real-time energy management approaches (i.e., myopic policy, model predictive control, and regular DRL approaches) demonstrate the effectiveness and superiority of the proposed approach.

Abstract:In a high-risk sector, such as power system, transparency and interpretability are key principles for effectively deploying artificial intelligence (AI) in control rooms. Therefore, this paper proposes a novel methodology, the evolving symbolic model (ESM), which is dedicated to generating highly interpretable data-driven models for dynamic security assessment (DSA), namely in system security classification (SC) and the definition of preventive control actions. The ESM uses simulated annealing for a data-driven evolution of a symbolic model template, enabling different cooperative learning schemes between humans and AI. The Madeira Island power system is used to validate the application of the ESM for DSA. The results show that the ESM has a classification accuracy comparable to pruned decision trees (DTs) while boasting higher global interpretability. Moreover, the ESM outperforms an operator-defined expert system and an artificial neural network in defining preventive control actions.

Abstract:We propose a quasi-deterministic proxy for the network-constrained stochastic unit commitment (SUC) problem. The proposed proxy can identify very similar commitment decisions as those obtained by solving the SUC problem with a large scenario set. Its computational performance, though, is close to that of a deterministic unit commitment problem. The proposed proxy has the same formulation as the SUC problem but only includes one or two envelope scenarios, generated based on the original scenario set. The two envelope scenarios capture the maximum and minimum net-load conditions in the original scenario set. We use a systematic method to assess the quality of commitment decisions obtained by the proposed proxy. The considered case study is based on the Illinois 200-bus system.

Abstract:Energy storage can smooth the fluctuations of wind power integrated into the grid. Due to the strong adaptability of the empirical mode decomposition (EMD) algorithm to non-stationary signals, it is widely used in wind power smoothing control strategies. However, traditional EMD algorithms cannot guarantee that the upper and lower areas of the calculated intrinsic mode functions (IMFs) are equal, which tends to result in imbalanced calculated energy storage power and thus exceeding the limit of energy storage capacity. Focusing on wind power smoothing control by energy storage, this paper proposes a strategy based on the area-equilibrium EMD, which modifies the upper and lower areas of the IMFs to achieve a more balanced distribution. As a result, the IMFs contain less energy, and consequently, the energy contained in the calculated smoothing power is also reduced. This makes the energy storage capacity less likely to exceed the limit, thereby achieving better wind power smoothing performance under given energy storage capacity. Case studies show that the proposed strategy results in more balanced upper and lower areas of the IMFs, reduces the fluctuating range of calculated energy storage, and improves the wind power smoothing effectiveness.

Abstract:In volt/var control (VVC) for active distribution networks, it is essential to integrate traditional voltage regulation devices with modern smart photovoltaic inverters to prevent voltage violations. However, model-based multi-device VVC methods rely on accurate system models for decision-making, which can be challenging due to the extensive modeling workload. To tackle the complexities of multi-device cooperation in VVC, this paper proposes a two-timescale VVC method based on reinforcement learning with hybrid action space, termed the hybrid action representation twin delayed deep deterministic policy gradient (HAR-TD3) method. This method simultaneously manages traditional discrete voltage regulation devices, which operate on a slower timescale, and smart continuous voltage regulation devices, which function on a faster timescale. To enable effective collaboration between the different action spaces of these devices, we propose a variational auto-encoder based hybrid action reconstruction network. This network captures the interdependencies of hybrid actions by embedding both discrete and continuous actions into the latent representation space and subsequently decoding them for action reconstruction. The proposed method is validated on IEEE 33-bus, 69-bus, and 123-bus distribution networks. Numerical results indicate that the proposed method successfully coordinates discrete and continuous voltage regulation devices, achieving fewer voltage violations compared with state-of-the-art reinforcement learning methods.

Abstract:In modern microgrids (MGs) with high penetration of distributed energy resources (DERs), system reconfiguration occurs more frequently and becomes a significant issue. Fixed-parameter controllers may not handle these tasks effectively, as they lack the ability to adapt to the dynamic conditions in such environments. This paper proposes an intelligence-driven grid-forming (GFM) converter control method for islanding MGs using a robustness-guided neural network (RNN). To enhance the adaptability of the proposed method, traditional proportional-integral controllers in the GFM primary control loops are entirely replaced by the RNN. The RNN is trained by a robustness-guided strategy to replicate their robust behaviors. All the training stages are purely data-driven methods, which means that no system parameters are required for the controller design. Consequently, the proposed method is an intelligence-driven model-less GFM converter control. Compared with traditional methods, the simulation results in all testing scenarios show the clear benefits of the proposed method. The proposed method reduces overshoots by more than 71.24%, which keeps all damping ratios within the stable region and provides faster stabilization. In comparison to traditional methods, at the highest probability, the proposed method improves damping by over 14.7% and reduces the rates of change of frequency and voltage by over 59.97%. Additionally, the proposed method effectively suppresses the interactions between state variables caused by inverter-based resources, with frequencies ranging from 1.0 Hz to 1.422 Hz. Consequently, these frequencies contribute less than 19.79% to the observed transient responses.

Abstract:Demand response (DR) is a practical solution to overcoming the challenges posed by the volatility and intermittency of the renewable generation in power systems. Industrial electricity demand is growing rapidly, which makes the DR potential estimation of industrial user critical for the DR implementation. In this paper, a unified model for estimating DR potential in the production processes of aluminum, cement, and steel is proposed on the basis of their unique operational characteristics. Firstly, considering the typical characteristic constraints of different industrial users, a DR potential estimation model is developed to capture typical industrial user response behavior under various operational and economic factors. The proposed estimation model is further refined to account for the uncertain and subjective factors present in the actual estimation environment. Secondly, a virtual data acquisition method is introduced to obtain the private virtual parameters required in the estimation process. Then, an industrial user participation threshold is presented to determine whether industrial users may participate in DR at a given time with consideration of their response characteristics. The industrial users may not always act with perfect rationality, and the response environment remains uncertain. In addition, the subjective factor in this paper includes the proposed threshold and the bounded rationality. Finally, an improved DR potential estimation model is proposed to reduce the difficulties in the actual estimation process. The simulation results validate the effectiveness of the proposed estimation model and the improved DR potential estimation model across multiple cases.

Abstract:Over the past decade, bidding in electricity markets has attracted widespread attention. Reinforcement learning (RL) has been widely used for electricity market bidding as a powerful artificial intelligence (AI) tool to make decisions under real-world uncertainties. However, current RL-based bidding methods mostly employ low-dimensional bids (LDBs), which significantly diverge from the N price-power pairs commonly used in current electricity markets. The N-pair bid format is denoted as high-dimensional bid (HDB) format, which has not been fully integrated into the existing RL-based bidding methods. The loss of flexibility of current RL-based bidding methods could greatly limit the bidding profits and make it difficult to address the increasing uncertainties caused by renewable energy generation. In this paper, we propose a framework for fully utilizing HDBs in RL-based bidding methods. First, we employ a special type of neural network called the neural network supply function (NNSF) to generate HDBs in the form of N price-power pairs. Second, we embed the NNSF into a Markov decision process (MDP) to make it compatible with most existing RL algorithms. Finally, the experiments on energy storage systems (ESSs) in the Pennsylvania-New Jersey-Maryland (PJM) real-time electricity market show that the proposed bidding method with HDBs can increase the bidding flexibility, thereby increasing the profits of state-of-the-art RL-based bidding methods.

Abstract:Power system optimal dispatch with transient security constraints is commonly represented as transient security-constrained optimal power flow (TSC-OPF). Deep reinforcement learning (DRL)-based TSC-OPF trains efficient decision-making agents that are adaptable to various scenarios and provide solution results quickly. However, due to the high dimensionality of the state space and action spaces, as well as the non-smoothness of dynamic constraints, existing DRL-based TSC-OPF solution methods face a significant challenge of the sparse reward problem. To address this issue, a fast-converging DRL method for optimal dispatch of large-scale power systems under transient security constraints is proposed in this paper. The Markov decision process (MDP) modeling of TSC-OPF is improved by reducing the observation space and smoothing the reward design, thus facilitating agent training. An improved deep deterministic policy gradient algorithm with curriculum learning, parallel exploration, and ensemble decision-making (DDPG-CL-PE-ED) is introduced to drastically enhance the efficiency of agent training and the accuracy of decision-making. The effectiveness, efficiency, and accuracy of the proposed method are demonstrated through experiments in the IEEE 39-bus system and a practical 710-bus regional power grid. The source code of the proposed method is made public on GitHub.

Abstract:Driven by increasing penetration of intermittent renewable energy generation, modern power systems are promoting the integration of energy storage (ES) and advocating high-resolution dynamic security constrained optimal power flow (DSCOPF) models to exploit ES time-shifting flexibility against contingencies and respond promptly to more frequent variations in the system operating status. While pioneering research works explore different methods to solve security constrained optimal power flow (SCOPF) problems at individual time steps, real-time implementation of DSCOPF still faces challenges associated with uncertainty adaptation, complex constraint satisfaction， and computational efficiency. This paper proposes a physics-guided safe policy learning method, featuring an analytical evaluation model to provide both accurate safety and cost-efficiency evaluations. A primal-dual-based learning procedure is developed to guide policy learning, fostering prompt convergence. A spatial-temporal graph neural network is constructed to enhance perception on the spatial-temporal uncertainties and leverage policy generalization. Case studies validate the effectiveness and scalability of the proposed method in safety, cost-efficiency, and computational performance and highlight the value of enhanced perception on IEEE 39-bus and 118-bus test systems.

Abstract:Short-term voltage stability (STVS) assessment is a critical monitoring technology in modern power systems. During daily operations, transmission lines may switch on or off due to scheduled maintenance or unexpected faults, which poses challenges to the STVS assessment under varying topology change conditions. To adapt the STVS assessment to the system topology changes, we propose a deep-learning-based STVS assessment model with the topology-adaptive voltage dynamic feature and the fine-tuning domain transfer for power systems with changing system topologies. The topology-adaptive voltage dynamic feature, extracted from streaming time-series data of phasor measurement units (PMUs), is used to characterize transient voltage stability. The voltage dynamic features depend on the balance of reactive power flow and system topology, effectively revealing both spatio-temporal patterns of post-disturbance system dynamics. The simulation results based on large disturbances in the New England 39-bus power system demonstrate that the proposed model achieves superior STVS assessment performance, with an accuracy of 99.65% in predicting voltage stability compared with the existing deep learning methods. The proposed model also performs well when applied to the larger IEEE 145-bus power system. The fine-tuning domain transfer of the proposed model adapts very well to system topology changes in power systems. It achieves an accuracy of 99.50% in predicting the STVS for the New England 39-bus power system with the transmission line alternation. Furthermore, the proposed model demonstrates strong robustness to noisy and missing data.

Abstract:The data-driven approaches have been extensively developed for multi-operation impedance modeling of the renewable power generation equipment (RPGE). However, due to the black box of RPGE, the dataset used for establishing impedance model lacks theoretical guidance for data generation, which reduces data quality and results in a large amount of data redundancy. To address this issue, this paper proposes an impedance dataset optimization method for data-driven modeling of RPGE considering multi-operation conditions. The objective is to improve the data quality of the impedance dataset, thereby reflecting the overall impedance characteristics with a reduced data amount. Firstly, the impact of operation conditions on impedance is evaluated to optimize the selection of operating points. Secondly, at each operating point, the frequency distribution is designed to reveal the impedance characteristics with fewer measurement points. Finally, a serial update method for measured datasets and the multi-operation impedance model is developed to further refine the dataset. The experiments based on control-hardware-in-loop (CHIL) are conducted to verify the effectiveness of the proposed method.

Abstract:A mathematical programming approach rooted in distributionally robust optimization (DRO) provides an effective data-driven strategy for battery energy storage system (BESS) planning. Nevertheless, the DRO paradigm often lacks interpretability in its results, obscuring the causal relationships between data distribution characteristics and the outcomes. Furthermore, the current approach to battery type selection is not included in traditional BESS planning, hindering comprehensive optimization. To tackle these BESS planning problems, this paper presents a universal method for BESS planning, which is designed to enhance the interpretability of DRO. First, mathematical definitions of interpretable DRO (IDRO) are introduced. Next, the uncertainties in wind power, photovoltaic power, and loads are modeled by using second-order cone ambiguity sets (SOCASs). In addition, the proposed method integrates selection, sizing, and siting. Moreover, a second-order cone bidirectional-orthogonal strategy is proposed to solve the BESS planning problems. Finally, the effectiveness of the proposed method is demonstrated through case studies, offering planners richer decision-making insights.

Abstract:As power systems scale up and uncertainties deepen, traditional centralized optimization approaches impose significant computation burdens on large-scale optimization problems, introducing new challenges for power system scheduling. To address these challenges, this study formulates a distributionally robust optimization (DRO) scheduling model that considers source-load uncertainty and is solved using a novel distributed approach that considers the distribution of tie-line endpoints. The proposed model includes a constraint related to the transmission interface, which consists of several tie-lines between two subsystems and is specifically designed to ensure technical operation security. In addition, we find that tie-line endpoints enhance the speed of distributed computation, leading to the development of a power system partitioning approach that considers the distribution of these endpoints. Further, this study proposes a distributed approach that employs an integrated algorithm of column-and-constraint generation (C&CG) and sub-gradient descent (IACS) to address the proposed model across multiple subsystems. A case study of two IEEE test systems and a practical provincial power system demonstrates that the proposed model effectively ensures system security. Finally, the scalability and effectiveness of the distributed approach in accelerating problem-solving are confirmed.

Abstract:With the increasing integration of uncertain distributed renewable energies (DREs) into distribution networks (DNs), communication bottlenecks and the limited deployment of measurement devices pose significant challenges for advanced data-driven voltage control strategies such as deep reinforcement learning (DRL). To address these issues, this paper proposes an offline-training online-execution framework for volt-var control in DNs. In the offline-training phase, a graph convolutional network (GCN)-based denoising autoencoder (DAE), referred to as the deep learning (DL) agent, is designed and trained to capture spatial correlations among limited physical quantities. This agent predicts voltage values for nodes with missing measurements using historical load data, DRE outputs, and global voltages from simulations. Furthermore, the dual-timescale voltage control problem is formulated as a multi-agent Markov decision process. A DRL agent employing the multi-agent soft actor-critic (MASAC) algorithm is trained to regulate the tap position of on-load tap changer (OLTC) and reactive power output of photovoltaic (PV) inverters. In the online-execution phase, the DL agent supplements the limited measurement data, providing enhanced global observations for the DRL agent. This enables precise equipment control based on improved system state estimation. The proposed framework is validated on two modified IEEE test systems. Numerical results demonstrate its ability to effectively reconstruct missing measurements and achieve rapid, and accurate voltage control even under severe measurement deficiencies.

Abstract:Wide-area damping controllers (WADCs) help in damping poorly damped inter-area oscillations (IAOs) using wide-area measurements. However, the vulnerability of the communication network makes the WADC susceptible to malicious dynamic attacks. Existing cyber-resilient WADC solutions rely on accurate power system models or extensive simulation data for training the machine learning (ML) model, which are difficult to obtain for large-scale power system. This paper proposes a novel non-intrusive hybrid two-stage detection framework that mitigates these limitations by eliminating the need for real-time access to large system data or attack samples for training the ML model. In the first stage, an autoencoder is deployed at the actuator location to detect dynamic attacks with sharp gradient variations, e.g., triangular, saw-tooth, pulse, ramp, and random attack signals. In the second stage, an unscented Kalman filter with unknown input estimation at the control center identifies smoothly varying dynamic attacks by estimating the control signal received by the actuator using synchrophasor measurements. A modified cosine similarity (MCS) metric is proposed to compare and quantify the similarity between the estimated control signal and the control signal sent by the WADC placed at the control center to detect any dynamic attacks. The MCS is designed to differentiate between events and dynamic attacks. The performance of the proposed framework has been validated on a hardware-in-the-loop (HIL) cyber-physical testbed built by using the OPAL-RT simulator and industry-grade hardware.

Abstract:As the scale of power system continues to grow, a fast and accurate distributed optimal power flow solver becomes crucial for the effective dispatch of power system. This paper presents a learning to optimize (L2O) approach to accelerating the distributed optimal power flow solving. The final convergence values of global variables and Lagrange multipliers of the alternating direction method of multipliers (ADMM) are estimated as its warm-start solution. A long short-term memory-variational auto-encoder (LSTM-VAE) model is developed as the core for estimating the convergence value, and the LSTM-VAE assisted ADMM is proposed. The LSTM generates low-dimensional representations of global variables and Lagrange multipliers, while the decoder part of VAE reconstructs the high-dimensional asymptotic convergence values. A novel loss function is designed in the form of a quadratic sum penalty term to incorporate the constraint violations of the Lagrange multipliers. Additionally, a two-stage training data generation strategy is proposed to efficiently generate substantial data within a limited amount of time. The effectiveness of the LSTM-VAE assisted ADMM is evaluated using the modified IEEE 123-bus system, a synthetic 500-bus system, and a 793-bus system.

Abstract:In this paper, policy-assisted graph reinforcement learning (PAGRL) is proposed for real-time economic dispatch (RTED). RTED is presented as a sequential decision problem formulated by Markov decision process (MDP). PAGRL employs a graph convolutional network to extract grid operation features containing topological information and then an agent that performs power dispatch is trained through proximal policy optimization. Moreover, the adaptiveness of agent to more hard-to-learn scenarios is enhanced by difficulty sampling, and policy-assisted action post-processing mechanism is designed to reduce search space and improve decision quality, which provides a general performance enhancement scheme for reinforcement learning in power system applications. Comparative studies on modified IEEE 118-bus system and real-world provincial grid demonstrate the flexible and reliable performance of the proposed PAGRL for RTED.

Abstract:This work presents a new topological observability algorithm to strengthen the performance of the hybrid power system static state estimation, assuming that the supervisory control and data acquisition (SCADA) and phasor measurement unit (PMU) measurements are recorded at the same time intervals. The observability of each estimated state variable is assessed by the value assigned to its least local redundancy index. The algorithm has been specifically developed to enhance the observability of an existing wide-area monitored system and exempts its expansion from critical sets and critical measurements. These objectives are achieved by building the incidence matrices of the measurements for the nodes and branches. The performance of the proposed algorithm is evaluated using the IEEE test systems and the SIN test systems of Brazilian equivalent systems.

Abstract:Power transfer limit (PTL) calculation plays an important role in assessing power network capability under certain constraints of system security and stability. However, the impact of ambient factors, which is different due to variations in time and space, is ignored in traditional methods to obtain PTL, thus inducing errors. Furthermore, system operation based on traditional PTL results may increase system security risks, particularly in the case of power flow congestion under heavy loads. Therefore, this paper proposes a decentralized PTL calculation method with improved optimal power flow model, which allows for the effect of the ambient factors characterized by the balance of heat absorption and dissipation for overhead conductors. The ambient factors of overhead transmission lines and the temperature of overhead conductors are involved as independent variables and state variables, respectively. Moreover, the sequential optimization problem is decomposed into several subproblems by the optimal conditional decomposition to deal with the temporal coupling constraints, and a parallel decomposition framework is used to solve multiple subproblems in parallel. Finally, the proposed method is implemented on two test systems under varying ambient factors, demonstrating the efficiency of the proposed method and the significant impacts of spatial and seasonal differences on PTL results.

Abstract:Optimal capacity configuration (OCC) of large-scale energy bases with multi-timescale operation characteristics presents a critical challenge. To address the problem, this study proposes an OCC approach of large-scale energy bases considering external multi-stochastic scenarios and interactive multi-timescale objectives. Firstly, guided by the system theory, the nonlinear state-space description is presented for systematic analysis of a general large-scale energy base. Due to interactive multi-timescale objectives between annual and daily cumulative objectives, a nested optimization structure is established. Then, considering the external multi-stochastic scenarios caused by the variables such as wind speed, solar irradiance, electric load, and thermal load, a multi-step optimization strategy is proposed including pre-configuration in regular scenarios and re-configuration by introducing micro-incremental scenarios. The multi-step optimization strategy and nested optimization structure jointly achieve the OCC of the large-scale energy base. In each step, the nested optimization structure is executed once. Finally, while ensuring the balance between thermal supply and load demand, the imbalances between electric power supply and the load demand are eliminated, significantly showing the efficiency of the proposed OCC approach.

Abstract:The large-scale integration of renewable energy sources, such as wind power and solar power, into the power system has significantly transformed its characteristics. The issue of sub-synchronous oscillation (SSO) becomes increasingly prominent, severely impacting the system stability. As the wind turbines vary in structures and parameters, existing model-based SSO suppression approaches do not fully consider wind turbine differences and multi-mode oscillation frequencies. To address these issues, this paper proposes a decentralized SSO suppression controller for doubly-fed induction generator (DFIG)-based wind farms using periodic updating data-enabled predictive control (PUDeePC) approach. Firstly, to better adapt to the time-varying system and external disturbance, a periodic updating algorithm is proposed incorporating anomaly detection. The stability of the PUDeePC approach is theoretically validated, and its robustness to variations and disturbances is qualitatively analyzed. Finally, the effectiveness of the PUDeePC approach is revealed through numerical simulations under various conditions, including compensation level variation, wind power output variation, number of online DFIG variations, multi-mode SSOs, and asynchronous PUDeePC approach.

Abstract:This paper introduces a machine learning (ML) based model predictive control (MPC) with piecewise-affine approximation (PWA) structure for maximizing wind energy capture for an individual wind turbine operating in wind farms with low-quality wind resources. While MPC has the capability to systematically consider the stochasticity of wind speed and the dynamic process of wind turbine, its real-time implementation in a hardware controller of wind turbine has not been successful due to its high online computational burden and stringent execution time requirement in practice. To address this long-standing issue, this paper proposes a two-phase ML-based method consisting of linear regression and clustering to construct a PWA of the optimal law for original MPC scheme. The two-phase ML-based method is tunable with computational complexity, which can be adjusted to meet the hardware limitation of the given controller of wind turbine to enable real-time implementation, while preserving the optimality of linearized full-fidelity MPC as much as possible. We conduct simulations and experiments to demonstrate the effectiveness of the two-phase ML-based method.

Abstract:Distribution networks have been experiencing significant changes under the pressure of the energy transition. The high integration of renewable energy sources combined with the electrification process introduces new challenges in managing distribution networks. Innovative solutions aimed at optimizing the control of complex problems, starting from historical data instead of a detailed system model, have been growing due to rapid development in artificial intelligence and machine learning. This paper proposes a Q-learning algorithm to control the tap setting of the on-load tap changer installed in primary substation transformers. The ultimate goal is to maintain voltage magnitudes at all busses of the medium-voltage distribution network within a safe range, simultaneously optimizing on-load tap changer operations. As a case study, the effectiveness of the proposed algorithm is assessed using a real medium-voltage distribution network with high penetration of renewable energy sources that supplies more than 2500 users/prosumers. The ability of the proposed algorithm to control bus voltages is tested in several scenarios characterized by significant variability and uncertainty. Outcomes show that the proposed algorithm is suitable for optimizing voltage control in distribution networks using a data-driven approach.

Abstract:This paper proposes a method for obtaining nonlinear models of distribution grid based on available measurements from the power grid. We formulate a maximum likelihood estimation (MLE) problem that estimates unknown line parameters—specifically, the impedance between nodes—using measured voltage magnitudes and net active and reactive power injections at each node. The nonlinear model for the distribution grid uses a nonlinear approximation of the DistFlow model, which includes line losses and is parameterized by the unknown line impedances. We solve the resulting MLE problem using an expectation maximization (EM) algorithm, tailored for the nonlinear model, and provide a numerically robust implementation. The proposed method is demonstrated on the IEEE 37-node test network, and we compare it with the state-of-the-art methods. The proposed method achieves a 70% reduction in voltage error and an error for state variables that is more than 10000 times smaller. A final comparison uses data from a real network, and the proposed method achieves parameter estimates with errors 100 times smaller than competing methods.

Abstract:With the increasing number of distributed flexible resources with energy storage capabilities in virtual power plants (VPPs), the traditional market clearing model that only includes quantity and price bids cannot fully unlock their potential flexibility. In light of this, we propose a market clearing model for energy-constrained virtual power plants (EC-VPPs) based on distributionally robust chance-constrained optimization (DRCCO) with moment information. Furthermore, to address the uncertainty of EC-VPPs in the electricity market, a pricing strategy for EC-VPPs is proposed. This strategy helps quantify the impact of uncertainty in EC-VPPs on the system economy. The proposed market clearing model is reformulated as a tractable mixed-integer second-order cone programming (MISOCP) problem via a two-sided distributionally robust chance-constrained convex reformulation method. Numerical simulations verify that the proposed pricing strategy offers fair incentives for both reserve providers and uncertain sources, and delivers an effective market mechanism for the EC-VPPs.