Abstract:This study investigates a safe reinforcement learning algorithm for grid-forming (GFM) inverter based frequency regulation. To guarantee the stability of the inverter-based resource (IBR) system under the learned control policy, a model-based reinforcement learning (MBRL) algorithm is combined with Lyapunov approach, which determines the safe region of states and actions. To obtain near optimal control policy, the control performance is safely improved by approximate dynamic programming (ADP) using data sampled from the region of attraction (ROA). Moreover, to enhance the control robustness against parameter uncertainty in the inverter, a Gaussian process (GP) model is adopted by the proposed algorithm to effectively learn system dynamics from measurements. Numerical simulations validate the effectiveness of the proposed algorithm.

Abstract:The high proportion of renewable energy integration and the dynamic changes in grid topology necessitate the enhancement of voltage/var control (VVC) to manage voltage fluctuations more rapidly. Traditional model-based control algorithms are becoming increasingly incompetent for VVC due to their high model dependence and slow online computation speed. To alleviate these issues, this paper introduces a graph attention network (GAT) based deep reinforcement learning for VVC of topologically variable power system. Firstly, combining the physical information of the actual power grid, a physics-informed GAT is proposed and embedded into the proximal policy optimization (PPO) algorithm. The GAT-PPO algorithm can capture topological and spatial correlations among the node features to tackle topology changes. To address the slow training, the ReliefF-S algorithm identifies critical state variables, significantly reducing the dimensionality of state space. Then, the training samples retained in the experience buffer are designed to mitigate the sparse reward issue. Finally, the validation on the modified IEEE 39-bus system and an actual power grid demonstrates superior performance of the proposed algorithm compared with state-of-the-art algorithms, including PPO algorithm and twin delayed deep deterministic policy gradient (TD3) algorithm. The proposed algorithm exhibits enhanced convergence during training, faster solution speed, and improved VVC performance, even in scenarios involving grid topology changes and increased renewable energy integration. Meanwhile, in the adopted cases, the network loss is reduced by 6.9%, 10.8%, and 7.7%, respectively, demonstrating favorable economic outcomes.

Abstract:The increasing penetration of renewable energy resources and reduced system inertia pose risks to frequency security of power systems, necessitating the development of fast frequency regulation (FFR) methods using flexible resources. However, developing effective FFR policies is challenging because different power system operating conditions require distinct regulation logics. Traditional fixed-coefficient linear droop-based control methods are suboptimal for managing the diverse conditions encountered. This paper proposes a dynamic nonlinear P-f droop-based FFR method using a newly established meta-reinforcement learning (meta-RL) approach to enhance control adaptability while ensuring grid stability. First, we model the optimal FFR problem under various operating conditions as a set of Markov decision processes and accordingly formulate the frequency stability-constrained meta-RL problem. To address this, we then construct a novel hierarchical neural network (HNN) structure that incorporates a theoretical frequency stability guarantee, thereby converting the constrained meta-RL problem into a more tractable form. Finally, we propose a two-stage algorithm that leverages the inherent characteristics of the problem, achieving enhanced optimality in solving the HNN-based meta-RL problem. Simulations validate that the proposed FFR method shows superior adaptability across different operating conditions, and achieves better trade-offs between regulation performance and cost than benchmarks.

Abstract:Building integrated energy systems (BIESs) are pivotal for enhancing energy efficiency by accounting for a significant proportion of global energy consumption. Two key barriers that reduce the BIES operational efficiency mainly lie in the renewable generation uncertainty and operational non-convexity of combined heat and power (CHP) units. To this end, this paper proposes a soft actor-critic (SAC) algorithm to solve the scheduling problem of BIES, which overcomes the model non-convexity and shows advantages in robustness and generalization. This paper also adopts a temporal fusion transformer (TFT) to enhance the optimal solution for the SAC algorithm by forecasting the renewable generation and energy demand. The TFT can effectively capture the complex temporal patterns and dependencies that span multiple steps. Furthermore, its forecasting results are interpretable due to the employment of a self-attention layer so as to assist in more trustworthy decision-making in the SAC algorithm. The proposed hybrid data-driven approach integrating TFT and SAC algorithm, i.e., TFT-SAC approach, is trained and tested on a real-world dataset to validate its superior performance in reducing the energy cost and computational time compared with the benchmark approaches. The generalization performance for the scheduling policy, as well as the sensitivity analysis, are examined in the case studies.

Abstract:In a high-risk sector, such as power system, transparency and interpretability are key principles for effectively deploying artificial intelligence (AI) in control rooms. Therefore, this paper proposes a novel methodology, the evolving symbolic model (ESM), which is dedicated to generating highly interpretable data-driven models for dynamic security assessment (DSA), namely in system security classification (SC) and the definition of preventive control actions. The ESM uses simulated annealing for a data-driven evolution of a symbolic model template, enabling different cooperative learning schemes between humans and AI. The Madeira Island power system is used to validate the application of the ESM for DSA. The results show that the ESM has a classification accuracy comparable to pruned decision trees (DTs) while boasting higher global interpretability. Moreover, the ESM outperforms an operator-defined expert system and an artificial neural network in defining preventive control actions.

Abstract:This paper demonstrates a new type of sub-synchronous oscillation of the grid-forming (GFM) converter, which occurs at low rather than high power levels. To reveal the intrinsic mechanism, a simplified analytical small-signal impedance model of the GFM converter is derived. It is found that the reactive power control loop (RPCL) can have a significant impact on the system stability. In particular, the voltage matrix introduced by the RPCL is the key factor causing instability in the GFM grid-connected system at low operating points. Therefore, this paper proposes a control strategy that reshapes the RPCL to counteract the negative effect of the voltage matrix by introducing a q-axis current feedforward, ensuring stable operation at any operating point. Finally, experimental results validate the correctness and effectiveness of the proposed control strategy.

Abstract:The probabilistic short circuit analysis provides relevant information for power system planning and power quality assessment tasks. Traditional Monte Carlo methods (TMCMs) are usually applied to consider the randomness affecting short circuit operating conditions, but they require numerous iterations to properly characterize the network conditions. This paper proposes a mixed Taguchi-based method (MTBM) as a new alternative tool to account for the uncertainties affecting the inputs of probabilistic short circuit analysis. The MTBM significantly reduces the number of iterations required to properly address the randomness of inputs (environmental conditions, pre-fault conditions, fault characteristics), and allows diversifying the representation of inputs through a quantile-based selection of their levels. The proposed method is applied to unbalanced three-phase four-wire low-voltage (LV) distribution systems with photovoltaic systems (PVSs) operating in low voltage ride- through (LVRT) during the fault. Numerical applications related to a test system are presented, and the proposed MTBM is compared with the TMCM, the unmixed Taguchi-based method (UTBM), and the point estimate method (PEM). The proposed MTBM returns values very close to those of the TMCM (with average deviations ranging from 0.01% to 3.12%) and enables a fast and accurate analysis of faulted LV distribution systems with PVSs operating in LVRT.

Abstract:Meter encoding, as a side-effect-free scheme, has been proposed to detect false data injection (FDI) attacks without significantly affecting the operation of power systems. However, existing meter encoding schemes either require encoding lots of measurements from different buses to protect a substantial proportion of a power system or are unhidden from alert attackers. To address these issues, this paper proposes a smart inverter enabled meter encoding scheme for detecting FDI attacks in distribution system state estimation. The proposed scheme only encodes the measurements from the existing programmable smart inverters. Meanwhile, this scheme can protect all the downstream buses from the encoded inverter bus. Compared with existing schemes, the proposed scheme encodes fewer meters when protecting the same number of buses, which decreases the encoding cost. In addition, by following the physical power flow laws, the proposed scheme is hidden from alert attackers who can implement the state estimation-based bad data detection (BDD). Simulation results from the IEEE 69-bus distribution system demonstrate that the proposed scheme can mislead the attacker’s state estimation on all the downstream buses from the encoded bus without arousing the attacker’s suspicion. FDI attacks that are constructed based on the misled estimated state are very likely to trigger the defender’s BDD alarm.

Abstract:To address the limitations of traditional planning methods in handling complex scenarios such as multi-feeder or substation cluster supply under high photovoltaic (PV) penetration, this paper proposes a collaborative configuration optimization method of soft open points (SOPs) and distributed multi-energy stations with spatiotemporal coordination and complementarity to reduce renewable energy curtailment. First, a shared strategy of multiple types of resources is proposed based on an SOP-enabled flexible distribution network. Second, a distributed hydrogen-based multi-energy coupling system (DHMECS) is developed. Then, a DHMECS siting model considering inter-feeder resource sharing is formulated. Finally, a configuration model of SOP and DHMECS is proposed, incorporating a partitioned autonomous operation strategy that considers spatiotemporal coordination and complementarity. The proposed method is validated on the improved Portugal 54-node and 219-node distribution networks, and the results demonstrate that it mitigates severe voltage violations and PV curtailment, enhances partitioned autonomous operation capabilities, and addresses the challenges of complex planning scenarios involving multi-feeder or substation cluster supply.

Abstract:With the development of advanced metering infrastructure (AMI), large amounts of electricity consumption data can be collected for electricity theft detection. However, the imbalance of electricity consumption data is violent, which makes the training of detection model challenging. In this case, this paper proposes an electricity theft detection method based on ensemble learning and prototype learning, which has great performance on imbalanced dataset and abnormal data with different abnormal level. In this paper, convolutional neural network (CNN) and long short-term memory (LSTM) are employed to obtain abstract feature from electricity consumption data. After calculating the means of the abstract feature, the prototype per class is obtained, which is used to predict the labels of unknown samples. In the meanwhile, through training the network by different balanced subsets of training set, the prototype is representative. Compared with some mainstream methods including CNN, random forest (RF) and so on, the proposed method has been proved to effectively deal with the electricity theft detection when abnormal data only account for 2.5% and 1.25% of normal data. The results show that the proposed method outperforms other state-of-the-art methods.

Abstract:Accurate prediction of the state-of-charge (SOC) of battery energy storage system (BESS) is critical for its safety and lifespan in electric vehicles. To overcome the imbalance of existing methods between multi-scale feature fusion and global feature extraction, this paper introduces a novel multi-scale fusion (MSF) model based on gated recurrent unit (GRU), which is specifically designed for complex multi-step SOC prediction in practical BESSs. Pearson correlation analysis is first employed to identify SOC-related parameters. These parameters are then input into a multi-layer GRU for point-wise feature extraction. Concurrently, the parameters undergo patching before entering a dual-stage multi-layer GRU, thus enabling the model to capture nuanced information across varying time intervals. Ultimately, by means of adaptive weight fusion and a fully connected network, multi-step SOC predictions are rendered. Following extensive validation over multiple days, it is illustrated that the proposed model achieves an absolute error of less than 1.5% in real-time SOC prediction.

Abstract:To secure power system operations, practical dispatches in industries place a steady power transfer limit on critical inter-corridors, rather than high-dimensional and strong nonlinear stability constraints. However, computational complexities lead to over-conservative pre-settings of transfer limit, which further induce undesirable and non-technical congestion of power transfer. To conquer this barrier, a scenario-classification hybrid-based banding method is proposed. A cluster technique is adopted to separate similarities from historical and generated operating condition dataset. With a practical rule, transfer limits are approximated for each operating cluster. Then, toward an interpretable online transfer limit decision, cost-sensitive learning is applied to identify cluster affiliation to assign a transfer limit for a given operation. In this stage, critical variables that affect the transfer limit are also picked out via mean impact value. This enables us to construct low-complexity and dispatcher-friendly rules for fast determination of transfer limit. The numerical case studies on the IEEE 39-bus system and a real-world regional power system in China illustrate the effectiveness and conservativeness of the proposed method.

Abstract:Rotor angle stability (RAS) prediction is critically essential for maintaining normal operation of the interconnected synchronous machines in power systems. The wide deployment of phasor measurement units (PMUs) promotes the development of data-driven methods for RAS prediction. This paper proposes a temporal and topological embedding deep neural network (TTEDNN) model to accurately and efficiently predict RAS by extracting the temporal and topological features from the PMU data. The grid-informed adjacency matrix incorporates the structural and electrical parameter information of the power grid. Both the small-signal RAS with disturbance under initial operating conditions and the transient RAS with short circuits on transmission lines are considered. Case studies of the IEEE 39-bus and IEEE 300-bus power systems are used to test the performance, scalability, and robustness against measurement uncertainties of the TTEDNN model. Results show that the TTEDNN model performs best among existing deep learning models. Furthermore, the superior transfer learning ability from small-signal RAS conditions to transient RAS conditions has been proved.

Abstract:In recent years, reinforcement learning (RL) has emerged as a solution for model-free dynamic programming problem that cannot be effectively solved by traditional optimization methods. It has gradually been applied in the fields such as economic dispatch of power systems due to its strong self-learning and self-optimizing capabilities. However, existing economic scheduling methods based on RL ignore security risks that the agent may bring during exploration, which poses a risk of issuing instructions that threaten the safe operation of power system. Therefore, we propose an improved proximal policy optimization algorithm for sequential security-constrained optimal power flow (SCOPF) based on expert knowledge and safety layer to determine active power dispatch strategy, voltage optimization scheme of the units, and charging/discharging dispatch of energy storage systems. The expert experience is introduced to improve the ability to enforce constraints such as power balance in training process while guiding agent to effectively improve the utilization rate of renewable energy. Additionally, to avoid line overload, we add a safety layer at the end of the policy network by introducing transmission constraints to avoid dangerous actions and tackle sequential SCOPF problem. Simulation results on an improved IEEE 118-bus system verify the effectiveness of the proposed algorithm.

Abstract:The operating conditions of wind turbines (WTs) in the same wind farm (WF) may share similarities due to their shared manufacturing process, control strategy, and operating environment. However, the similarities of WTs are seldom considered in WT anomaly detection, resulting in the disregard of useful information. This paper proposes a method to improve the reliability and accuracy of WT anomaly detection using the supervisory control and data acquisition (SCADA) data of multiple WTs in the same WF. First, a similarity assessment method based on a comparison of different observation time series is proposed, which objectively quantifies the similarities of WT operating conditions. Then, the SCADA data of the target WT and selected WTs that are similar are used to establish several estimation models through a long short-term memory (LSTM) algorithm. LSTM models that exhibit good estimation performance are used to construct a combined estimation model that estimates the variations in the monitored variables of the target WT. Finally, an anomaly detection method that jointly compares the effective value and information entropy of the residuals is proposed to identify anomalies. The effectiveness and accuracy of the proposed method are verified using the data of two actual WFs.

Abstract:Reactive power optimization of distribution networks is traditionally addressed by physical model based methods, which often lead to locally optimal solutions and require heavy online inference time consumption. To improve the quality of the solution and reduce the inference time burden, this paper proposes a new graph attention networks based method to directly map the complex nonlinear relationship between graphs (topology and power loads) and reactive power scheduling schemes of distribution networks, from a data-driven perspective. The graph attention network is tailored specifically to this problem and incorporates several innovative features such as a self-loop in the adjacency matrix, a customized loss function, and the use of max-pooling layers. Additionally, a rule-based strategy is proposed to adjust infeasible solutions that violate constraints. Simulation results on multiple distribution networks demonstrate that the proposed method outperforms other machine learning based methods in terms of the solution quality and robustness to varying load conditions. Moreover, its online inference time is significantly faster than traditional physical model based methods, particularly for large-scale distribution networks.

Abstract:The increasing integration of intermittent renewable energy sources (RESs) poses great challenges to active distribution networks (ADNs), such as frequent voltage fluctuations. This paper proposes a novel ADN strategy based on multi-agent deep reinforcement learning (MADRL), which harnesses the regulating function of switch state transitions for the real-time voltage regulation and loss minimization. After deploying the calculated optimal switch topologies, the distribution network operator will dynamically adjust the distributed energy resources (DERs) to enhance the operation performance of ADNs based on the policies trained by the MADRL algorithm. Owing to the model-free characteristics and the generalization of deep reinforcement learning, the proposed strategy can still achieve optimization objectives even when applied to similar but unseen environments. Additionally, integrating parameter sharing (PS) and prioritized experience replay (PER) mechanisms substantially improves the strategic performance and scalability. This framework has been tested on modified IEEE 33-bus, IEEE 118-bus, and three-phase unbalanced 123-bus systems. The results demonstrate the significant real-time regulation capabilities of the proposed strategy.

Abstract:Recently, we demonstrated the success of a time-synchronized state estimator using deep neural networks (DNNs) for real-time unobservable distribution systems. In this paper, we provide analytical bounds on the performance of the state estimator as a function of perturbations in the input measurements. It has already been shown that evaluating performance based only on the test dataset might not effectively indicate the ability of a trained DNN to handle input perturbations. As such, we analytically verify the robustness and trustworthiness of DNNs to input perturbations by treating them as mixed-integer linear programming (MILP) problems. The ability of batch normalization in addressing the scalability limitations of the MILP formulation is also highlighted. The framework is validated by performing time-synchronized distribution system state estimation for a modified IEEE 34-node system and a real-world large distribution system, both of which are incompletely observed by micro-phasor measurement units.

Abstract:The intermittency of renewable energy generation, variability of load demand, and stochasticity of market price bring about direct challenges to optimal energy management of microgrids. To cope with these different forms of operation uncertainties, an imitation learning based real-time decision-making solution for microgrid economic dispatch is proposed. In this solution, the optimal dispatch trajectories obtained by solving the optimal problem using historical deterministic operation patterns are demonstrated as the expert samples for imitation learning. To improve the generalization performance of imitation learning and the expressive ability of uncertain variables, a hybrid model combining the unsupervised and supervised learning is utilized. The denoising autoencoder based unsupervised learning model is adopted to enhance the feature extraction of operation patterns. Furthermore, the long short-term memory network based supervised learning model is used to efficiently characterize the mapping between the input space composed of the extracted operation patterns and system state variables and the output space composed of the optimal dispatch trajectories. The numerical simulation results demonstrate that under various operation uncertainties, the operation cost achieved by the proposed solution is close to the minimum theoretical value. Compared with the traditional model predictive control method and basic clone imitation learning method, the operation cost of the proposed solution is reduced by 6.3% and 2.8%, respectively, over a test period of three months.

Abstract:In recent years, the expansion of the power grid has led to a continuous increase in the number of consumers within the distribution network. However, due to the scarcity of historical data for these new consumers, it has become a complex challenge to accurately forecast their electricity demands through traditional forecasting methods. This paper proposes an innovative short-term residential load forecasting method that harnesses advanced clustering, deep learning, and transfer learning technologies to address this issue. To begin, this paper leverages the domain adversarial transfer network. It employs limited data as target domain data and more abundant data as source domain data, thus enabling the utilization of source domain insights for the forecasting task of the target domain. Moreover, a K-shape clustering method is proposed, which effectively identifies source domain data that align optimally with the target domain, and enhances the forecasting accuracy. Subsequently, a composite architecture is devised, amalgamating attention mechanism, long short-term memory network, and seq2seq network. This composite structure is integrated into the domain adversarial transfer network, bolstering the performance of feature extractor and refining the forecasting capabilities. An illustrative analysis is conducted using the residential load dataset of the Independent System Operator to validate the proposed method empirically. In the case study, the relative mean square error of the proposed method is within 30 MW, and the mean absolute percentage error is within 2%. A significant improvement in accuracy, compared with other comparative experimental results, underscores the reliability of the proposed method. The findings unequivocally demonstrate that the proposed method advocated in this paper yields superior forecasting results compared with prevailing mainstream forecasting methods.

Abstract:Fault diagnosis (FD) for offshore wind turbines (WTs) are instrumental to their operation and maintenance (O&M). To improve the FD effect in the very early stage, a condition monitoring based sample set mining method from supervisory control and data acquisition (SCADA) time-series data is proposed. Then, based on the convolutional neural network (CNN) and attention mechanism, an interpretable convolutional temporal-spatial attention network (CTSAN) model is proposed. The proposed CTSAN model can extract deep temporal-spatial features from SCADA time-series data sequentially by ① a convolution feature extraction module to extract features based on time intervals； ② a spatial attention module to extract spatial features considering the weights of different features; and ③ a temporal attention module to extract temporal features considering the weights of intervals. The proposed CTSAN model has the superiority of interpretability by exposing the deep temporal-spatial features extracted in a human-understandable form of the temporal-spatial attention weights. The effectiveness and superiority of the proposed CTSAN model are verified by real offshore wind farms in China.

Abstract:As numerous distributed energy resources (DERs) are integrated into the distribution networks, the optimal dispatch of DERs is more and more imperative to achieve transition to active distribution networks (ADNs). Since accurate models are usually unavailable in ADNs, an increasing number of reinforcement learning (RL) based methods have been proposed for the optimal dispatch problem. However, these RL based methods are typically formulated without safety guarantees, which hinders their application in real world. In this paper, we propose an RL based method called supervisor-projector-enhanced safe soft actor-critic (S3AC) for the optimal dispatch of DERs in ADNs, which not only minimizes the operational cost but also satisfies safety constraints during online execution. In the proposed S3AC, the data-driven supervisor and projector are pre-trained based on the historical data from supervisory control and data acquisition (SCADA) system, effectively providing enhanced safety for executed actions. Numerical studies on several IEEE test systems demonstrate the effectiveness and safety of the proposed S3AC.

Abstract:Lately, the power demand of consumers is increasing in distribution networks, while renewable power generation keeps penetrating into the distribution networks. Insufficient data make it hard to accurately predict the new residential load or newly built apartments with volatile and changing time-series characteristics in terms of frequency and magnitude. Hence, this paper proposes a short-term probabilistic residential load forecasting scheme based on transfer learning and deep learning techniques. First, we formulate the short-term probabilistic residential load forecasting problem. Then, we propose a sequence-to-sequence (Seq2Seq) adversarial domain adaptation network and its joint training strategy to transfer generic features from the source domain (with massive consumption records of regular loads) to the target domain (with limited observations of new residential loads) and simultaneously minimize the domain difference and forecasting errors when solving the forecasting problem. For implementation, the dominant techniques or elements are used as the submodules of the Seq2Seq adversarial domain adaptation network, including the Seq2Seq recurrent neural networks (RNNs) composed of a long short-term memory (LSTM) encoder and an LSTM decoder, and quantile loss. Finally, this study conducts the case studies via multiple evaluation indices, comparative methods of classic machine learning and advanced deep learning, and various available data of the new residentical loads and other regular loads. The experimental results validate the effectiveness and stability of the proposed scheme.

Abstract:This paper highlights the inefficiency of most distributed controls in dealing with dynamic enhancement while coordinating distributed generators (DGs), leading to poor frequency dynamics. To address this concern, a two-level coupling-based frequency control strategy for microgrids is proposed in this paper. At the lower level, an adaptive dynamic compensation algorithm is designed to tackle short-term and long-term frequency fluctuations caused by the uncertainties of renewable energy resources (RESs). At the upper level, an adaptive distributed frequency consensus algorithm is developed to address frequency restoration and active power sharing. Furthermore, to account for the potential control interaction of the two designed levels, a nonlinear extended state observer (NESO) is introduced to couple their control dynamics. Simulation tests and hardware-in-the-loop (HIL) experiments confirm the improved frequency dynamics.

Abstract:The inter-regional electricity market is instrumental in enhancing the economic efficiency, reliability, and integration of renewable generation within interconnected power systems. As the market boundary expands, the complexity and solution difficulties of market clearing increase rapidly. The presence of hybrid alternating current (AC)/direct current (DC) interconnector networks further compounds challenges in modeling trading paths and transmission tariffs. To address these issues, this paper proposes a path-aware market-clearing (PAMC) model tailored for the inter-regional electricity market, which accommodates the hybrid AC/DC interconnector network. A variable aggregation strategy is proposed to reduce the problem scale while ensuring equivalent optimality. In addition, a novel redundancy elimination method is developed to expedite the solution of the market-clearing problem. This framework utilizes envelope approximations of residual demand curves to identify bidding blocks that will not affect the marginal price. Corresponding decision variables are then constrained to their bounds to remove redundant information. Comprehensive case studies across different power system scales validate the superiority of the proposed PAMC model in improving social welfare, and verify the effectiveness of the proposed redundancy elimination method in accelerating the solution of the market-clearing problem.

Abstract:Photovoltaic (PV) generation always exhibits strong uncertainty and variability; therefore, its excessive integration brings huge risks to the safe operation of power systems. In this letter, a two-stage robust optimization approach based on decision-dependent uncertainty is devised to identify the PV hosting capacity that can be accepted to ensure the effective consumption of PV generation under uncertainty. The proposed approach is validated by numerical experiments for a microgrid and a distribution network.