Abstract
This paper proposes a robust and computationally efficient control method for damping ultra-low frequency oscillations (ULFOs) in hydropower-dominated systems. Unlike the existing robust optimization based control formulation that can only deal with a limited number of operating conditions, the proposed method reformulates the control problem into a bi-level robust parameter optimization model. This allows us to consider a wide range of system operating conditions. To speed up the bi-level optimization process, the deep deterministic policy gradient (DDPG) based deep reinforcement learning algorithm is developed to train an intelligent agent. This agent can provide very fast lower-level decision variables for the upper-level model, significantly enhancing its computational efficiency. Simulation results demonstrate that the proposed method can achieve much better damping control performance than other alternatives with slightly degraded dynamic response performance of the governor under various types of operating conditions.
IN recent years, the negative effects of the fossil fuel based power system attract more and more attentions. In this context, increasing the penetration of renewable energy with low carbon and sustainable characteristics in the power system is a significant pathway to address this issue [
In [
Optimizing the proportional-integral-derivative (PID) parameters of the governor is another alternative for ULFO control. It is also shown that the re-tuning of the governor settings has high practical value [
To this end, a novel bi-level PID parameter optimization model is proposed for ULFO control. It has the following contributions:
1) Based on the Routh-Hurwitz criterion, the mechanism of the ULFO is studied and the feasibility of formulating ULFO control as optimizing PID parameters is demonstrated. In particular, the problem of ULFO control is formulated as a bi-level robust PID optimization model. This is in contrast with the formulation in [
2) To improve the efficiency of solving bi-level model, this paper forms the lower-level model into the Markov decision process (MDP) solved by a deep deterministic policy gradient (DDPG) based algorithm. After that, the decision variables transferred by the upper-level model can be quickly addressed via the well-trained DDPG agent without the repeated optimization. This is a novel method to solve min-max optimization model and is different from previous min-max model [
The rest of this paper is organized as follows. Section II introduces the system model. In Section III, the formulation of governor parameter optimization is presented. The proposed bi-level robust parameter optimization model is presented in Section IV. The case study is provided in Section V. Section VI presents the actual hydropower-dominated system and conclusions are given in Section VII.
(1) |

Fig. 1 Schematic of two-machine system.
where is the inertia time constant; is the rotor speed deviation; is the mechanical power deviation; is the electromagnetic power deviation; and is the damping coefficient.
Assuming frequency-dependent load as , where is the load frequency sensitivity, and substituting into (1), the transfer function of the generator can be described as [
(2) |
where . The transfer function of prime mover (governor and turbine) can be defined as [
(3) |
where and are the transfer functions of governor and turbine, respectively; , , and are the proportional, integral, and differential coefficients of the governor, respectively; is the adjustment coefficient of the governor; is the response time of the governor; and is the water hammer effect and it depends on the operating conditions of hydroturbine [
(4) |
where and are the length and sectional area of the diversion pipeline, respectively; is the rated water flow; is the gravitational acceleration; is the rated water head; and is the number of diversion pipelines.
For the thermal power unit, this paper ignores the boiler dynamic process and mainly considers the steam turbine and governor [
(5) |
where is the damping torque of thermal generator; is the transfer function of thermal power unit; is the gain; and are the time-steps of hydraulic system and high-pressure cylinder, respectively.
The dynamic responses of G1 and G2 under fault 1, i.e., a double-phase short-circuit fault at bus 3 from 2.0 s to 2.2 s, are shown in

Fig. 2 Dynamic responses of G1 and G2 under fault 1.
To investigate the cause of ULFO in the studied system, we calculate the damping torque coefficient of two units’ primary frequency regulation (PFR). Among them, the detailed calculation process of damping torque coefficient of hydropower unit is provided in [
(6) |
Submitting into (3), the damping torque coefficient of stream power unit can be obtained as [
(7) |
(8) |
where denotes the damping torque coefficient; and denotes the synchronous torque coefficient.
Submitting into (6) and (8), the trajectory of damping torque coefficients and changing with frequency f can be obtained, as shown in

Fig. 3 Trajectory of damping torque coefficients of hydropower and stream power units. (a) Hydropower unit. (b) Stream power unit.
It can be observed that the hydropower unit will produce a negative damping under the ultra-low frequency band (below 0.1 Hz). In contrast, the thermal power unit would produce a positive damping in this band. Therefore, the ULFO is strongly related with hydropower unit.
To further investigate the relationship of ULFO and hydropower unit, the characteristic equation of closed-loop transfer function of the hydropower unit is calculated based on (2) and (3) and can be written as:
(9) |
After simplification, (9) can be rewritten as:
(10) |
where a1, a2, a3, and a4 are the Routh-Hurwitz criterion coefficients, which are related to . Specifically, the Routh-Hurwitz criterion coefficients and oscillation frequency changing with are shown in

Fig. 4 Routh-Hurwitz criterion coefficients and oscillation frequency changing with Tw. (a) Routh-Hurwitz criterion coefficients. (b) Oscillation frequency.
It can be observed from
In summary, the ULFO is strongly related to the hydro-power units, and it is caused by the PFR of the hydropower units. More specifically, due to the water hammer effect, the hydraulic governor easily produces negative damping torque, as shown in
Besides, Routh-Hurwitz criterion coefficients are related to the hydrogovernor PID parameters. In fact, we can adjust PID parameter settings to make these coefficients keep positive. Based on this consideration, we test the trajectory of c1 with different PID parameter settings and the results are shown in

Fig. 5 c1 changing with different PID parameter settings.
Based on the system linearization technology, the state matrix A can be obtained [
(11) |
(12) |
where and are the left and right feature vectors, respectively; is the diagonal matrix; denotes the
As mentioned above, optimizing PID parameter settings contributes to stable ULFO mode . However, there are some requirements: ① less effect on other oscillation modes: we should avoid weakening the damping of other modes when improving the damping of ULFO mode; ② good dynamic performance for the PFR: previous studies indicate that if without proper design, optimizing PID parameters may deteriorate PFR dynamic performance, and thus weakening the frequency adjustment ability of governor [
(13) |
where is the objective function of the optimization problem; is the time constant of water hammer effect of the governor representing the operating conditions of the governor and limited between 0.5 and 3 [
(14) |
where is the stability of the ULFO; JITAE is the primary frequency control performance of the prime mover; is the desired real part of the eigenvalues; is the damping ratio of the eigenvalues; is the dynamic response of governor under disturbance; and is the steady-state value of the prime mover under disturbance. In fact, can be further written as:
(15) |
It can be observed from (15) that, when , JSTB is defined as , by minimizing this target, and of ULFO mode would be pushed close to the predetermined damping and real part. Note that the damping of ULFO mode is set to be a bigger value. By optimizing PID parameter settings to make the damping of ULFO mode close to the predetermined damping, the damping of ULFO mode is improved. Once the damping and real parts of ULFO mode reach to the predetermined value, JSTB becomes 0.
By minimizing , the output deviation of the prime mover can be optimized to close to steady-state point. In this way, both the oscillation amplitude and oscillation time of the prime mover under disturbance can be optimized, which means that the dynamic performance of the primary frequency control can be improved.
For the bi-level robust parameter optimization model, the lower-level model is reformulated as the MDP and solved by the DDPG algorithm. Subsequently, the trained agent is employed to assist particle swarm optimization (PSO) to solve the upper-level model to obtain the optimal solution. The overall scheme of the optimization process is shown in

Fig. 6 Overall scheme of optimization process.
The core functions of lower-level model shown in
(16) |
In each episode, the upper-level model would provide a set of PID parameter setting to the lower-level model. The lower-level model searches the extreme operating condition with the worst performance under this PID setting condition. In this way, both the upper-level and lower-level variables are determined and the objective function (14) can be calculated and sent to the upper-level model.
In fact, the function of the lower-level model is to find the corresponding extreme operating condition under every PID setting condition. Therefore, the key to solving this lower-level model is to find the policy function that maximizes the objective function. It is a decision-making problem in an uncertain environment. This paper reformulates it as an MDP, and the key elements related to MDP are defined as a tuple, , where S denotes the state and is composed of the PID parameter settings of each hydrogovernor; A denotes the action and is represented as the operation conditions of governor; P denotes the transition probability; and R denotes the reward and is used to evaluate the action taken by the agent at each time step. In this paper, the reward is defined as:
(17) |
where the superscript k denotes the
The reinforcement learning algorithm is a common method to solve this MDP [
1) Q-learning: it is a widely used reinforcement learning method [
(18) |
where is the discount rate; and is the learning rate. The greedy policy is adopted in Q-learning, where the agent will choose the action with the highest Q-value during the training process. Note that Q-learning is only suitable for the case where state-action space is small. The increase of state-action space size would make the Q-table become too big, resulting in each Q-value being rarely updated. In this context, deep Q-network (DQN) is proposed to address it.
2) DQN: DQN is proposed for solving high dimension state-action problem via combining deep neural network (DNN) with Q-learning method. Specifically, a DNN is utilized to approximate Q-table, named Q-network, represented as , where denotes the parameters of DNN. The Q-network takes the state as input and outputs the Q-value for each state-action pair. It can be trained via minimizing the loss function:
(19) |
where is the expected function; denotes the target Q-value, and denotes the parameter of target Q-network and is updated via soft-update method [
Moreover, a replay buffer is employed in DQN to break the identically distributed state samples and reduce the correlation between them, leading to improved data efficiency. Specifically, during the training process, all information is saved as an experience and stored in the memory . After the buffer is full, the oldest experience will be replaced by the newly obtained one. Subsequently, at each iteration, the agent will sample a mini-batch of experience from the replay buffer.
3) DDPG: DDPG is the standard DQN method only working effectively for solving control problems with continuous states and low-dimension discrete action sets. It is not suitable to solve the optimization problem of PID parameter setting. To this end, DDPG is introduced and it can achieve better performance in solving control problems with continuous action space than DQN.
During the training process, these two networks are trained against each other. Among them, the critic network can be updated by the loss function [
(20) |
where is the number of mini-batch; and is the learning rate of the critic networks, Then, the parameters of the actor network can be updated by the gradient descent [
(21) |
where is the learning rate of the actor network; and is the Gaussian noise.
Moreover, to stabilize the training process, both target actor-critic pair networks are added in DDPG, which can be parameterized by and . At each iteration, the soft updating (22) is utilized to synchronize its parameters to the target actor-critic networks [
(22) |
where is the soft-update rate.
After off-line training, the well-trained DDPG-enabled agent can learn the optimal control policy and provide extreme operating conditions for each PID parameter setting of the system.
(23) |
In this way, the proposed bi-level min-max optimization model (13) can be transformed into a single-level mathematical programming problem with constraints, i.e.,
(24) |
It can be observed from (23) that the bi-level model is converted into a nonlinear optimization problem. A heuristic algorithm is a good choice to solve that and this paper uses the PSO algorithm [
Step 1: define the solution space and fitness function. The PSO is used to find the optimal PID parameter settings for governor. In this context, the particle position is designed as the PID parameter settings . The fitness function is applied to evaluate the training error and the goodness of a given solution, which is defined as the objective function and shown in (14).
Step 2: initialize random swarm location and velocities. Before beginning to search the optimal position, each particle is initialized with the random PID parameter setting within the allowable ranges. Moreover, the direction and length of movement of the particle at each episode are named velocity, which is also initialized.
Step 3: calculate the fitness of each particle. The PID parameter setting carried by each particle is transmitted to the well-trained DDPG agent. Then, the actor network in DDPG can provide the extreme operating condition with the worst performance for each particle under the corresponding PID parameter setting. Next, both PID parameter setting and operating condition data of each particle can be updated to the studied system to calculate the fitness via (14).
Step 4: update the particle position and velocity. The velocity and position of each particle can be updated via:
(25) |
where is the velocity of the particle at the iteration; is the position of the particle at the iteration; is the best position among all particles in the population up to the iteration; is the optimal position of the particle up to the iteration; is the inertia factor; and c2 are the acceleration coefficients; and r1 and r2 are the random numbers in . In this paper, ; [
Step 5: iterate to find the optimal solution. Repeat Steps 2 and 3 until the minimum error is met, or the maximum number of iterations is reached. Output the final result as the solution to the above optimization problem.
In this section, the performance of the proposed method is tested on the IEEE 10-machine 39-bus system. All generators adopt a fifth-order model and a simplified excitation system [
The numbers of layers and neurons for the networks in the DDPG algorithm are set as follows: both actor and critic networks adopt the same structure, which contains three hidden layers; and the numbers of neurons for the hidden layers are 128, 64, and 64, respectively. The hyperparameters of the DDPG algorithm can be found in
Parameter | Value |
---|---|
Learning rate for actor network | 0.001 |
Learning rate for critic network | 0.002 |
Experience replay memory capacity | 8000 |
Step size of each episode | 12 |
Mini-batch size | 40 |
To obtain the corresponding extreme operating conditions with different PID parameter settings, which maximize the objective function, numerous scenarios are constructed for training. Specifically, we generate scenarios by randomly sampling from the upper- and lower-limits of PID parameter settings of each hydroturbine (, , ). Note that 95% of these PID parameter settings are used for training while the remaining 5% are used for testing. The cumulative reward changing with the episode during the training process is shown in

Fig. 7 Cumulative reward changes with episode during training process.
After training, the well-trained agent can provide each PID parameter setting to the corresponding extreme operating condition. After that, this agent can be applied to assist the PSO algorithm to solve the upper-level model. To show the computational efficiency of this agent, we compare the solution methods of the lower-level model in [
Method | Average solution time of lower-level model (s) |
---|---|
Optimization method in [ | 13.40 |
Optimization method in [ | 3.70 |
Proposed method | 0.05 |
Moreover, we further investigate the performance of the PSO and differential evolution (DE) to test the performance of these two methods. To balance the training time and optimization effect, by careful trial and error, the population size of these two methods is set to be 30. Except the population size, the hyperparameters of the method have a great impact on the optimization performance. To deal with it, each hyperparameter in the method will first be discretized into several points in its feasible region. Then, these discrete points form a test table by the orthogonal method.
In this way, we can form nine sets of hyperparameter settings for DE and PSO algorithms, respectively. Then, we can train the PSO and DE algorithms with each set of hyperparameter settings, as shown in

Fig. 8 Convergence process of DE and PSO algorithms with different hyperparameter settings.
Algorithm | Value of quantitative indicator | |||
---|---|---|---|---|
Best | Average | Worst | Standard deviation | |
PSO | 26.42 | 33.18 | 53.30 | 9.71 |
DE | 25.01 | 41.07 | 140.27 | 37.68 |
To test the effectiveness of the proposed method, five cases are tested by setting different time constants of the hydro-turbine , as shown in
Generator | of hydro-turbine | ||||
---|---|---|---|---|---|
Case 1 | Case 2 | Case 3 | Case 4 | Case 5 | |
G1 | 0.5 | 2.5 | 1.5 | 1.5 | 2.5 |
G2 | 1.5 | 1.5 | 2.5 | 0.5 | 0.5 |
G3 | 0.5 | 1.5 | 1.5 | 1.5 | 0.5 |
G4 | 2.5 | 2.5 | 2.5 | 1.5 | 1.5 |
G5 | 1.5 | 2.5 | 0.5 | 1.5 | 0.5 |
G6 | 1.5 | 0.5 | 0.5 | 1.5 | 2.5 |
G7 | 0.5 | 1.5 | 0.5 | 0.5 | 2.5 |
G8 | 0.5 | 0.5 | 1.5 | 1.5 | 1.5 |
G9 | 1.5 | 2.5 | 1.5 | 2.5 | 1.5 |
G10 | 2.5 | 0.5 | 1.5 | 0.5 | 2.5 |

Fig. 9 Frequency deviations of units under fault 1. (a) G1. (b) G2.
Method | Real part | Imaginary part | Damping (%) |
---|---|---|---|
Original parameters | 0.0005 | 0.54 | -0.093 |
Robust method I | -0.0410 | 0.59 | 6.930 |
Robust method II | -0.0540 | 0.55 | 9.770 |
Proposed method | -0.0920 | 0.52 | 17.420 |
To further elaborate on the results, the dynamic characteristic of G1 and step responses of PFR are shown in Figs.

Fig. 10 Dynamic characteristic of G1. (a) Damping curve. (b) Step response.

Fig. 11 Step responses of PFR. (a) G1. (b) G2.
To test the robustness of all three methods under different disturbances, four different two-phase short-circuit faults are applied with 100 ms fault duration and the faulty buses are 8, 16, 23, and 25, which are named as fault 2, fault 3, fault 4, and fault 5, respectively. The simulation results are shown in

Fig. 12 Dynamic response of G1 under different disturbances. (a) Fault 2. (b) Fault 3. (c) Fault 4. (d) Fault 5.
Furthermore, different Tw settings for Cases 2-5 in
For each case, time-domain simulations are carried out with different PID parameter settings. The results are shown in

Fig. 13 Dynamic response of G1 under different cases. (a) Case 2. (b) Case 3. (c) Case 4. (d) Case 5.
It can be concluded that the PID parameters optimized by the proposed method can achieve better performance in comparison with two other optimization methods. To further investigate the performance of the proposed method, we adopt the Monte Carlo method to sample time constant Tw of turbines between 0.5 s and 3 s to generate 300 cases. For each case, the ULFO mode is calculated and the probability density function (PDF) of ULFO nodes is also calculated, as shown in

Fig. 14 Damping PDF of ULFO mode under different cases. (a) Original parameters. (b) Robust method I. (c) Robust method II. (d) Proposed method.
Method | Mean value (%) | Standard deviation |
---|---|---|
Original parameters | 0.27 | 0.0075 |
Robust method I | 6.18 | 0.0420 |
Robust method II | 10.25 | 0.0536 |
Proposed method | 13.78 | 0.0213 |
In this section, an actual hydropower-dominated system in Sichuan Power Grid, China is introduced as a studied system, which is shown in

Fig. 15 Simplified system topology used for case.
To test the effectiveness of the proposed method under different datasets, we generate three datasets via different distribution functions. Specifically, we generate three datasets by sampling from the upper- and lower-limits of PID parameter settings of each hydroturbine via the uniform distribution function, normal distribution function, and rayleigh distribution function, respectively. Each dataset is utilized to train a DDPG agent. The training process is shown in

Fig. 16 Cumulative reward changes with episode under different datasets.
It can be observed from
To test the control performance of obtained PID parameter settings, we send the PID parameter settings to the system and compare these settings under a fault, where a two-phase short-circuit faults are applied with 150 ms fault duration and the faulty buses are CJL-220. The simulation is carried out under two different cases and the results are shown in

Fig. 17 Comparison results of different PID parameter settings obtained via different datasets. (a) . (b) .
Moreover, to further investigate the performance of the proposed method, the common PID parameter tuning method (Ziegler-Nichols) is applied to act as benchmark method. The details of this method can be found in [

Fig. 18 Comparison results of different PID tuning methods. (a) . (b) .
In this paper, a novel bi-level robust parameter optimization model is proposed to re-tune the PID parameters to control ULFO. Different from the conventional robust optimization methods, the PID parameter optimization is reformulated into the form of a min-max optimization model to ensure the effectiveness of the optimized PID parameters under various extreme operating conditions. The DDPG is developed to train an agent for the fast decision making of lower-level model and ensure the effective interactions with the upper-level model, significantly improving the computational efficiency.
After the agent is trained, each iteration only needs 0.05 s to provide actions for the upper-level model. Simulation results are carried out in IEEE 10-machine 39-bus system and actual hydropower-dominated system in Sichuan Power Grid. Three hundred cases are generated by Monte Carlo method to act as test cases and the comparison results show that the proposed method can achieve better damping control performance. The mean value of the ULFO mode damping under 300 cases can reach 13.78%. The mean value of other two robust methods can only reach 6.18% and 10.25%, respectively. It means that the PID parameters optimized by the proposed method can achieve better damping control performance in comparison with two other robust optimization methods under different operating conditions.
Appendix
The standard PFR model of the hydropower unit is shown in Fig. A1.

Fig. A1 Structure diagram of standard PFR model of hydropower unit.
The parameter settings of hydropower unit, thermal power unit, and AC lines are listed in Tables AI-AIII, respectively. The parameter setting of generators in two-machine system is listed in Table AIV. The rated active and reactive power of generators and load are listed in Table V.
Variable | Value | Variable | Value |
---|---|---|---|
KP | 4.0 | bp | 0.05 |
KI | 2.5 | D | 2.00 |
KD | 0.5 | TG | 0.20 |
TJ | 10.0 | Tw | 4.00 |
Variable | Value |
---|---|
Ka | 1.0 |
Tch | 0.2 |
Tg | 0.3 |
AC line | R | X | B/2 |
---|---|---|---|
x1 | 0.0057 | 0.0625 | 0.5145 |
x2 | 0.0032 | 0.0323 | 0.2806 |
Parameter | G1 | G2 |
---|---|---|
Xd | 0.4400 | 1.0870 |
0.2500 | 0.2890 | |
0.2200 | 0.2020 | |
Xq | 1.7000 | 0.6840 |
0.5000 | 0.6870 | |
0.6000 | 0.2280 | |
X2 | 0.0025 | 0.2150 |
Ra | 0.0250 | 0 |
8.0000 | 10.0300 | |
0.0300 | 0.0400 | |
0.4000 | 0.2590 | |
0.0500 | 0.0600 | |
H | 13.000 | 12.0000 |
D | 0 | 0 |
Generator and load | P (MW) | Q (Mvar) |
---|---|---|
G1 | 700 | 185 |
G2 | 700 | 185 |
Load 1 | 600 | 100 |
The detailed expressions of Routh-Hurwitz criterion coefficients a1, a2, a3, and a4 are shown as follows:
(A1) |
The calculation process of oscillation frequency of two-machine system can be given.
Considering two-machine system is unstable, a dominant oscillation mode exists in the system. In this context, (5) can be represented as:
(A2) |
That is further rewritten as:
(A3) |
where , , and are the poles of the closed-loop transfer function. Note that and can be two real roots or a pair of the conjugate complex root. is the dominant pole. When , the frequency response will oscillate critically. Meanwhile, we have:
(A4) |
Based on (10), the oscillation frequency is defined as:
(A5) |
Combining (A5) and (A1), can be rewritten as:
(A6) |
The obtained PID parameters of IEEE 10-machine 39-bus system is shown in Table AVI.
Method | Generator | KP | KI | KD |
---|---|---|---|---|
Robust method I | G1 | 8.36 | 0.34 | 2.33 |
G2 | 2.77 | 0.04 | 1.18 | |
G5 | 17.90 | 0.09 | 2.66 | |
G8 | 1.54 | 0.18 | 1.10 | |
Robust method II | G1 | 8.61 | 0.29 | 2.69 |
G2 | 2.81 | 0.26 | 1.74 | |
G5 | 11.98 | 0.32 | 0.60 | |
G8 | 6.59 | 0.05 | 1.63 | |
Proposed method | G1 | 6.73 | 0.33 | 2.07 |
G2 | 2.28 | 0.11 | 1.39 | |
G5 | 13.58 | 0.18 | 1.52 | |
G8 | 2.68 | 0.21 | 1.73 |
References
Z. Arthouros. (2019, Feb.). The renewables 2019 global status report. [Online]. Available: http://www.ren21.net/gsr-2019/pages/foreword/fore word/ [Baidu Scholar]
C. Jiang, J. Zhou, P. Shi et al., “Ultra-low frequency oscillation analysis and robust fixed order control design,” International Journal of Electrical Power & Energy Systems, vol. 104, pp. 269-278, Jan. 2019. [Baidu Scholar]
W. Yang, J. Yang, R. Tang et al., “Experimental investigation of theoretical stability regions for ultra-low frequency oscillations of hydropower generating systems,” Energy, vol. 186, pp. 1-11, Nov. 2019. [Baidu Scholar]
Z. Liu, W. Yao, J. Wen et al., “Effect analysis of generator governor system and its frequency mode on inter-area oscillations in power systems,” International Journal of Electrical Power & Energy Systems, vol. 96, pp. 1-10, Mar. 2018. [Baidu Scholar]
W. Mo, Y. Chen, Y. Liu et al., “Analysis and measures of ultralow-frequency oscillations in a large-scale hydropower transmission system,” IEEE Journal of Emerging & Selected Topics in Power Electronics, vol. 6, no. 3, pp. 1077-1085, Sept. 2018. [Baidu Scholar]
H. Pico, J. D. McCalley, A. Angel et al., “Analysis of very low frequency oscillations in hydro-dominant power systems using multi-unit modeling,” IEEE Transactions on Power Systems, vol. 27, no. 4, pp. 1906-1915, Nov. 2012. [Baidu Scholar]
R. Xie, I. Innocent, D. Rimorov et al., “Fundamental study of common mode small-signal frequency oscillations in power systems,” International Journal of Electrical Power & Energy Systems, vol. 106, pp. 201-209, Jan. 2019. [Baidu Scholar]
Y. Chen, Y. Liu, Z. Tang et al., “Analysis of ultra-low frequency oscillation in yunnan asynchronous sending system,” in Proceedings of IEEE PES General Meeting, Chicago, USA, Jul. 2017, pp. 1-5. [Baidu Scholar]
G. Wang, X. Zheng, X. Guo et al., “Mechanism analysis and suppression method of ultra-low-frequency oscillations caused by hydropower units,” International Journal of Electrical Power & Energy Systems, vol. 103, pp. 102-114, Dec. 2018. [Baidu Scholar]
H. N. Villegas, “Electromechanical oscillations in hydro-dominant power systems: an application to the Colombian power system,” M.S. thesis, Department of Electrical and Computer Engineering, Iowa State University, Ames, USA, 2011. [Baidu Scholar]
K. Sebaa, Y. Zhou, Y. Li et al., “Low-frequency oscillation damping control for large-scale power system with simplified virtual synchronous machine,” Journal of Modern Power Systems and Clean Energy, vol. 9, no. 6, pp. 1424-1435, Nov. 2021. [Baidu Scholar]
G. Zhang, W. Hu, D. Cao et al., “Deep reinforcement learning based approach for proportional resonance power system stabilizer to prevent ultra-low-frequency oscillations,” IEEE Transactions on Smart Grid, vol. 11, no. 6, pp. 5260-5272, Nov. 2020. [Baidu Scholar]
R. Grondin, I. Kamwa, L. Soulieres et al., “An approach to PSS design for transient stability improvement through supplementary damping of the common low-frequency,” IEEE Transactions on Power Systems, vol. 8, no. 3, pp. 954-963, Aug. 1993. [Baidu Scholar]
P. Puri and S. Ghosh, “A hybrid optimization approach for PI controller tuning based on gain and phase margin specifications,” Swarm & Evolutionary Computation, vol. 8, pp. 69-78, Feb. 2013. [Baidu Scholar]
G. Chen, F. Tang, H. Shi et al., “Optimization strategy of hydro-governors for eliminating ultralow-frequency oscillations in hydro-dominant power systems,” IEEE Journal of Emerging & Selected Topics in Power Electronics, vol. 6, no. 3, pp. 1086-1094, Sept. 2018. [Baidu Scholar]
L. Chen, X. Lu, Y. Min et al., “Optimization of governor parameters to prevent frequency oscillations in power systems,” IEEE Transactions on Power Systems, vol. 33, no. 4, pp. 4466-4474, Jul. 2018. [Baidu Scholar]
D. J. White and G. Anandalingam, “A penalty function approach for solving bi-level linear programs,” Journal of Global Optimization, vol. 3, no. 4, pp. 397-419, Jul. 1993. [Baidu Scholar]
D. Bertsimas, E. Litvinov, X. Sun et al., “Adaptive robust optimization for the security constrained unit commitment problem,” IEEE Transactions on Power Systems, vol. 28, no. 1, pp. 52-63, Feb. 2013. [Baidu Scholar]
Y. Du, L. Jiang, Y. Li et al., “A robust optimization approach for demand side scheduling considering uncertainty of manually operated appliances,” IEEE Transactions on Smart Grid, vol. 9, no. 2, pp. 743-755, Mar. 2018. [Baidu Scholar]
S. Mei, W. Guo, Y. Wang et al., “A game model for robust optimization of power systems and its application,” Proceedings of the CSEE, vol. 33, no. 19, pp. 47-56, Jul. 2013. [Baidu Scholar]
M. Netto and L. Mili, “Robust data filtering for estimating electromechanical modes of oscillation via the multichannel prony method,” IEEE Transactions on Power Systems, vol. 33, no. 4, pp. 4134-4143, Jul. 2018. [Baidu Scholar]
G. Zhang, W. Hu, D. Cao et al., “A novel deep reinforcement learning enabled sparsity promoting adaptive control method to improve the stability of power systems with wind energy penetration,” Renewable Energy, vol. 178, pp. 363-376, Nov. 2021. [Baidu Scholar]
L. Bo, L. Han, C. Xiang et al., “A Q-learning fuzzy inference system based online energy management strategy for off-road hybrid electric vehicles,” Energy, vol. 252, pp. 1-11. Aug. 2022. [Baidu Scholar]
Z. Yan and Y. Xu, “Data-driven load frequency control for stochastic power systems: a deep reinforcement learning method with continuous action search,” IEEE Transactions on Power Systems, vol. 34, no. 2, pp. 1653-1656, Mar. 2019. [Baidu Scholar]
J. Duan, D. Shi, R. Diao et al., “Deep-reinforcement-learning-based autonomous voltage control for power grid operations,” IEEE Transactions on Power Systems, vol. 35, no. 1, pp. 814-817, Jan. 2020. [Baidu Scholar]
Y. Li, W. Gao, S. Huang et al., “Data-driven optimal control strategy for virtual synchronous generator via deep reinforcement learning approach,” Journal of Modern Power Systems and Clean Energy, vol. 9, no. 4, pp. 919-929, Jul. 2021. [Baidu Scholar]
G. Zhang, W. Hu, D. Cao et al., “A data-driven approach for designing STATCOM additional damping controller for wind farms,” International Journal of Electrical Power & Energy Systems, vol. 117, p. 105620, May 2020. [Baidu Scholar]
J. B. Park, K. S. Lee, J. R. Shin et al., “A particle swarm optimization for economic dispatch with non-smooth cost functions,” IEEE Transactions on Power Systems, vol. 20, no. 1, pp. 34-42, Feb. 2005. [Baidu Scholar]
H. Chen and Z. Xu, “Comparison of mathematical models for transient stability calculation in PSASP and PSS/E and corresponding calculation results,” Power System Technology, vol. 28, no. 5, pp. 1-12, Mar. 2004. [Baidu Scholar]
J. G. Ziegler and N. B. Nichols, “Optimum setting for automatic controllers,” Journal of Dynamic Systems Measurement and Control, vol. 64, pp. 759-768. Nov. 1942. [Baidu Scholar]