Abstract
The sequential method is easy to integrate with existing large-scale alternating current (AC) power flow solvers and is therefore a common approach for solving the power flow of AC/direct current (DC) hybrid systems. In this paper, a high-performance graph computing based distributed parallel implementation of the sequential method with an improved initial estimate approach for hybrid AC/DC systems is developed. The proposed approach is capable of speeding up the entire computation process without compromising the accuracy of result. First, the AC/DC network is intuitively represented by a graph and stored in a graph database (GDB) to expedite data processing. Considering the interconnection of AC grids via high-voltage direct current (HVDC) links, the network is subsequently partitioned into independent areas which are naturally fit for distributed power flow analysis. For each area, the fast-decoupled power flow (FDPF) is employed with node-based parallel computing (NPC) and hierarchical parallel computing (HPC) to quickly identify system states. Furthermore, to reduce the alternate iterations in the sequential method, a new decoupled approach is utilized to achieve a good initial estimate for the Newton-Raphson method. With the improved initial estimate, the sequential method can converge in fewer iterations. Consequently, the proposed approach allows for significant reduction in computing time and is able to meet the requirement of the real-time analysis platform for power system. The performance is verified on standard IEEE 300-bus system, extended large-scale systems, and a practical 11119-bus system in China.
THE rapidly-increasing capacity of modern power systems for power transmission has resulted in a pressing need to build large-scale alternating current (AC)/direct current (DC) hybrid systems with high-voltage DC (HVDC) transmission lines. The needs for high voltage level (over ±500 kV) and large DC power (over 3000 MW) in ultra-high voltage power systems [
1) In the unified method, the variables and equations of AC and DC grids are calculated simultaneously.
2) In the sequential method, the AC and DC equations are solved separately in a sequential process via inner and outer loops.
The most obvious advantage of the sequential method is the flexibility of implementing DC power flow analysis in the existing mature AC analysis, while, in the unified method, the solver must be re-programmed to accommodate the DC network. Thus, the sequential method is more flexible and suitable for solving large-scale hybrid systems. However, this method can be time-consuming because it calculates AC and DC networks repeatedly until achieving a global convergence, making it difficult to use in a real-time EMS, especially when the system is extremely large.
To reduce the computing time of the sequential method, there are two methods to be considered in the existing literature. The first method focuses on reducing the total amount of iterations. In [
The second method aims at improving the computing speed of power flow analysis. Some parallel approaches for solving a large sparse matrix, improving the Newton-Raphson method, and expediting data processing are proposed in [
In the process of solving large-scale data-driven problems, graph computing has shown its parallel efficiency due to the protogenetic parallel database in modeling problems, and the underlying parallel algorithms based on vertex/edge [
A graph computing based distributed parallel implementation of the sequential method with an improved initial estimate approach to speed up the analysis of large-scale AC/DC systems without compromising accuracy is presented in this paper. First, the AC/DC system is modeled in a GDB. Then, it is divided into smaller AC grids by using graph partitions to disconnect DC links. Slack buses are provided for all power grids and used to synchronize the results. In addition, power flow via the corresponding DC lines are equivalently allocated as extra power injections. Hence, the AC power flow of the inner calculating loops in the sequential method can be performed parallelly. Meanwhile, by decoupling the AC/DC connections in an improved way, a good initial estimate can be achieved to reduce the total number of iterations in the sequential method. In addition, fast-decoupled power flow (FDPF) and data processing with node-based parallel computing (NPC) and hierarchical parallel computing (HPC) are employed to speed up both inner and outer calculating loops to speed up the entire calculating process.
The rest of this paper is organized as follows. Section II describes graph computing and its applications in solving power system problems. Section III presents the power flow models of hybrid AC/DC system and the general computing processes of the sequential method. Section IV proposes the improved initial estimate approach and explains how the proposed method is applied in the sequential method with improvements. Section V presents the test results and discusses the value of the proposed method, and Section VI concludes this paper.
Graph theory is an advanced area of mathematical study which models complex networks and performs complicated calculations between the objects using “graphs”. Relevant graph-based techniques have been documented to significantly improve the general computing performance [

Fig. 1 Comparison of one-line and graph models for IEEE 14-bus system. (a) Single-line model. (b) Graph model.
The components constituting a conventional power grid can be classified as either vertex-attached or edge-attached. For example, generators, loads, static var compensators (SVCs) and converters together with their parameters are vertex-attached. The topology and attributes of transmission lines, transformers, filters and breakers are edge-attached. Thus, the graph model includes the information of both topology connections and related attributes. To illustrate the GDB in power system storage, the IEEE 5-bus system is used as an example.
As shown in

Fig. 2 Storage of power system modeling in two databases.
After modeling and storing a power system in a GDB, graph computing algorithms can be implemented and naturally executed on the underlying level with high parallel efficiency. The following subsections will introduce three graph-based algorithms and demonstrate their applications in solving power flow analysis.
In graph theory, computing can be executed on vertices via graph traversal, which is a seamless integration of the GDB. Since each vertex is capable of conducting local computing independently with information from itself and its neighbors, it is feasible and efficient to divide one problem into smaller sub-problems, and then conduct parallel calculations on each vertex. One of the examples is matrix formulation with NPC.
As shown in

Fig. 3 Application of NPC in processing admittance matrix.
Compared with NPC, which executes parallel calculations at each vertex, the HPC method assigns vertices to different levels based on the ordering of their connections. It performs parallel computing for vertices at the same level and solves the problem level by level in a hybrid sequential-parallel manner. One of the main applications is the LU factorization [

Fig. 4 Graph LU factorization. (a) Graph of matrix At. (b) Fill-ins determination. (c) Elimination tree and its partitions.
Considering iterations in the sequential method for AC/DC hybrid systems, the LU factorization would be performed several times before the convergence. Thus, an efficient HPC algorithm is extremely helpful.
The previous subsections have presented the concept and advantages of graph computing, but graph computing solves the system as an entity and performs the calculation and analysis together. Thus, all the vertices have to do the same computation locally in the beginning, then communicate and wait for synchronization in the end.
To expedite the calculation, the connecting topology of the hybrid system is taken into consideration to partition the original power network for distributed computing. Since most HVDC links are built to connect several disconnected AC grids crossing long distances, it is highly feasible to partition the network by disconnecting DC connections.
As shown in

Fig. 5 Example of system partition by disconnecting DC lines. (a) Without AC connections between areas. (b) With AC connections.
Again, commercially-available graph computing packages built on top of GDBs have implemented parallel computing based on NPC and HPC to optimally utilize underlying multi-cores, which are commonly-available computing architecture today. Thus, end users can transparently apply these techniques in power flow analysis. By contrast, traditional parallel methods cannot automatically take advantage of multi-core computing architecture. Further, after system partitioning, the overall network is decomposed into several independent smaller AC grids, such that each grid can be solved in a multi-core architecture. This makes the sequential method for hybrid AC/DC power flow a natural fit for distributed computing.
These are the key reasons that the proposed approach will best utilize multi-core architecture and achieve significant performance acceleration with graph computing.
The typical topology of DC links has been studied in detail. In general, the hybrid system can be modeled in three parts: AC grids, DC grids, and stations. AC grids are connected to stations via points of common coupling (PCC) and DC grids are connected via converters.

Fig. 6 Power exchange model of simplified AC/DC system.
In steady-state analysis, the equations of power balance can be utilized to describe the model [
(1) |
where and are the initialized or given active and reactive power on the buses, respectively; is the angle difference between buses i and j; and are the real and imaginary parts of the admittance matrix, respectively. In addition, the power injections at PCC can be calculated by:
(2) |
where . Furthermore, the generalized power loss of the station can be modeled as:
(3) |
where , , and are the coefficients that can be achieved by field testing. For the DC side, since only active power can be transferred through DC lines, the power balance is given by:
(4) |
Beyond the equations of power balance, the relationship of converter voltage and DC voltage for the line commutated converter (LCC) can also be given by:
(5) |
where is the number of bridges; and is the firing angle. Although the reactive power cannot be controlled directly by the LCC, the value can be calculated by:
(6) |
where is the power factor. The stations can be equalized to PQ or PV buses depending on different control strategies.
As stated in the previous subsection, a typical hybrid system can be decoupled in three parts and then analyzed separately. Power flow analysis is performed in each part, and then the results at PCC and station side are compared and updated for the next calculation. By conducting this process in sequence, global convergence can be achieved when the results calculated from AC sides and DC sides equal out. The sequential method must complete power flow analysis in two loops as shown in

Fig. 7 Inner and outer loops of sequential method.
Since it is difficult to measure accurate injection power at PCC as AC and DC grid input, the inner and outer loops are performed repeatedly to upgrade the values until global convergence is achieved. The power flows for the AC and DC grids are performed in the inner loops. Subsequently, the global convergence and value update are executed in the outer loops. An integral computing cycle includes an inner loop and an outer loop. In general, the sequential method is flexible and effective for use with large-scale hybrid systems with complex DC connections. By decoupling the system, modifications, and changes on the DC side such as converter blocks, transformer adjustments, and the switch of operating strategies, will not affect AC power flow, which is the main part of the calculation. Due to the multiple iterations in the two calculating loops, however, this does increase computing time. Therefore, new methods and tools to reduce the total number of iterations and expedite the overall calculation are necessary.
In the conventional sequential method, the uncertainties of power injections and power losses can greatly increase the total number of iterations before achieving global convergence. As discussed in Section III, power injections and losses at stations cannot be determined ahead of the calculation and should be calculated twice from both the AC and DC sides. After that, the results are compared and updated in the outer loops. Therefore, the calculation of the station values is independent of the inner loops of power flow analysis. Furthermore, since the convergence performance of the Newton-Raphson method highly depends on the initial estimate, if an initial estimate that is sufficiently close to the roots is given, the number of iterations can be greatly reduced. Therefore, the computing time can be significantly reduced if a good initial estimate can be provided instead of using a flat start.
Note that in

Fig. 8 Modified AC grid with new decoupled method.
Assuming that the original AC grid consists of n buses including PCC, by putting the station into the AC grid, the new modified AC grid is added with one branch and one bus afterwards. Thus, the new grid consists of buses. Furthermore, since is approximately equal to , the new bus can be set as a load which only absorbs active power. Consequently, the active power and power losses become parts of the AC grid and can be calculated together with AC power flow. In addition, since the topology of AC grids does not change, the results can be solved quickly via the FDPF method.
After the initial estimate is achieved, the main sequential method can be further expedited by implementing graph computing based techniques.
The conventional sequential method can be time-consuming due to multiple cycles of power flow calculation, equation formulation, PCC values updates, etc. To speed up the overall calculation without altering the main algorithms or compromising accuracy, graph computing based methods including graph partitioning, NPC, and HPC are employed, as shown in the flowchart in

Fig. 9 Flow chart of graph computing based distributed parallel AC/DC power flow with improved initial estimate approach.
To further illustrate how the graph partition is employed, a general model is given in

Fig. 10 Partition for generating distributed areas.
1) Type 1: isolated areas with pure AC components. After disconnecting DC lines, the sub-area is only composed of AC components, and isolated from other areas such as Area 2 and Area 3.
2) Type 2: connected AC areas with tie-line. The disconnection of DC lines does not divide the connected areas. In practical projects, long-distance high-voltage alternating current (HVAC) and HVDC lines may coexist to enhance power transmission capacity. Thus, Type 2 can be joined, generating a new larger area such as in Area 1 in
In the example of DC 3 branch, A1 and A2 are interconnecting and also connecting buses 4 and 5. During the process of partitioning, DC 3 branch is disconnected, and the power on DC converters are replaced by extra power injections at buses 4 and 5, i.e., . The rest of the DC links are equivalently replaced as extra load injections. After the system is partitioned and equivalent power is injected, new area slack buses are selected for the isolated areas (buses 1, 2 and 3 in
(7) |
where and are the sub-matrices of the approximated Jacobian matrix; and are the incremental vectors of bus voltage magnitude and phase angle, respectively; and and are the mismatching vectors of active and reactive power divided by corresponding bus voltage magnitude, respectively. The following text will further explain how NPC and HPC are implemented to improve the calculation.
1) NPC: formulating power flow equation (matrices and vectors), checking convergence and updating values. Considering that and are approximated Jacobian matrices, the diagonal elements represent the buses in the network and non-zero off-diagonal elements depict the existing connections between buses. In addition, each row vector is related to a corresponding bus in the system. In each row vector, non-zero off-diagonal elements are linked edges from the corresponding node and the diagonal element represents the node itself. Consequently, all the elements of the matrices can be generated locally and independently, indicating the feasible application of NPC. The right-hand side vectors in (1) are updated similarly. Thus, all power flow equations can be formulated using NPC. With the exception of equation formulation, convergence checks and status updates can also be conducted locally.
2) HPC: solving power flow. The sequential method requires multiple processes of FDPF; thus, an efficient solver is able to quickly find the solution. As shown in
The experiments are performed on a Linux server, with installation of a programmable GDB platform, TigerGraph. The server has a Xeon E7-8867 (V3 2.50 GHz) CPU and 64 GB memory with 16 cores which will be the platform for implementing the distributed AC/DC power flow algorithm. The operation system is CentOS 6.8 and the GDB version is 2.2. To guarantee precision, the iterative tolerance of both inner and outer loops is set as 1
To verify the accuracy and demonstrate the computing performance of the proposed method, the standard IEEE 300-bus system, two extended South Carolina 500-bus systems [
Compared with the extended ideal systems, which can be fully partitioned, the 11119-bus case is complicated by two factors. First, due to the coexistence of HVAC and HVDC in long-distance transmission, some areas are still connected even after disconnecting DC lines. Second, the operation parameters of the DC lines are diverse from each other, as shown in
To further illuminate the difference of structures, the admittance matrices of the 6000-bus case and the 11119-bus case are shown in

Fig. 11 Admittance matrices of different cases. (a) 6000-bus case. (b) 11119-bus case.
The commercial software PSS/E (v 33.0) is used for accuracy comparison. Since the equivalent power injections from DC lines to AC grids are much larger than other normal loads, the results at the coupling AC/DC buses are most likely to have the biggest differences. The test results also indicate that the biggest differences between PSS/E and the proposed method occurs at the DC linked buses. Thus, we provide the power flow results of these buses, together with DC operation values, as shown in Tables
The biggest differences of voltage magnitude, phase angle, and triggering angle are 0.00001 p.u., 0.0006°, and 0.004°, respectively, which are negligible in practical applications. The comparisons for the large-scale systems with multiple DC lines are listed in
To demonstrate the speedup in computing performance, four algorithms are compared, including the graph computing-based method (GC), the conventional sequential method (CONV), the default unified method and the modified sequential method in PSS/E (UPSS and SPSS). All the tests are performed on the same environment.

Fig. 12 Comparison of detailed running time of main calculating procedures.
As discussed previously, the major improvements are in the areas of graph partition, NPC, and HPC. Specifically, throughout the entire calculation, graph partitioning is only performed once at the beginning. A 10000-bus system can be completely calculated within 50 ms. Thus, the level of improvement justifies the extra cost.
Further, the computing time for formulating equations and performing FDPF is compared in
Based on
Another advantage of graph computing is the efficient utilization of multiple cores. The parallel algorithms based on vertices and edges can be programmed in the underlying multi-core architecture with a graph model. Thus, parallel cores can perform local parallel calculations directly.

Fig. 13 Comparison of detailed computing time with multiple cores.
While the parallel performance of HPC can benefit from the use of a greater number of cores, the number and size of partitioned areas also affect the real computing speed. When the same partition areas have fewer buses, as in the 6000-bus and 12000-bus cases, more cores (~16) can be utilized to achieve better speedup. In the 11119-bus case, since the partitioned areas are fewer and the largest area has a much higher number of buses, the best performance is achieved by using 12 cores. In all cases, when a greater number of cores are put into use, the overhead time spent on allocating cores, reading memory, and transmitting data will also increase.
As discussed in the previous sections, one of the advantages of a graph computing-based method is that each vertex in the graph model can be used as an independent logic and computing unit in the process of parallel computing. The proposed method performs better when used for larger systems, and it is therefore necessary to test the speedup ratios on larger systems. In this subsection, several extended large-scale cases based on South Carolina 500-bus systems are tested, and the computing time is compared using the CONV method. Note that these extended cases have similar connecting topologies as the 6000-bus and 12000-bus cases. The results are listed in
The comparison results in
In this paper, a parallel power flow method using a GDB and graph computing methods are developed for hybrid AC/DC systems to improve computing performance. After the network is modeled and stored in a GDB, the vertex and edge based parallel computing can be performed naturally and directly. With graph partitioning, the original large-scale system can be split into multiple independent AC grids, which are fit for distributed processing. Further, the graph computing techniques based on NPC and HPC, which efficiently utilize the underlying multi-core computing architecture, are implemented in the sequential method for AC/DC hybrid power flow to speed up the main time-consuming processes of formulating equations and solving FDPF. Simulation results on large-scale systems with over 10000 buses show significant speedup of the proposed method without compromising accuracy. Furthermore, the performances of multi-cores on graph partitioning, NPC and HPC also demonstrate high parallel efficiency.
REFERENCES
J. M. Maza-Ortega, E. Acha, S. García et al., “Overview of power electronics technology and applications in power generation transmission and distribution,” Journal of Modern Power Systems and Clean Energy, vol. 5, no. 4, pp. 499-514, Jul. 2017. [百度学术]
W. Long and S. Nilsson, “HVDC transmission: yesterday and today,” IEEE Power and Energy Magazine, vol. 5, no. 2, pp. 22-31, Mar.-Apr. 2007. [百度学术]
T. An, C. Han, Y. Wu et al., “HVDC grid test models for different application scenarios and load flow studies,” Journal of Modern Power Systems and Clean Energy, vol. 5, no. 2, pp. 262-274, Mar. 2017. [百度学术]
N. M. Haleem and A. D. Rajapakse, “Application of new directional logic to improve DC side fault discrimination for high resistance faults in HVDC grids,” Journal of Modern Power Systems and Clean Energy, vol. 5, no. 4, pp. 560-573, Jul. 2017. [百度学术]
P. Palensky and D. Dietrich, “Demand side management: demand response, intelligent energy systems, and smart loads,” IEEE Transactions on Industrial Informatics, vol. 7, no. 3, pp. 381-388, Aug. 2011. [百度学术]
J. Lei, T. An, Z. Du et al., “A general unified AC/DC power flow algorithm with MTDC,” IEEE Transactions on Power Systems, vol. 32, no. 4, pp. 2837-2846, Jul. 2017. [百度学术]
J. Beerten, S. Cole, and R. Belmans, “Generalized steady-state VSC MTDC model for sequential AC/DC power flow algorithms,” IEEE Transactions on Power Systems, vol. 27, no. 2, pp. 821-829, May 2012. [百度学术]
J. R. Silve and C. P. Arnold, “A simple improvement to sequential AC/DC power flow algorithms,” International Journal of Electrical Power & Energy Systems, vol. 12, no.3, pp. 291-221, Jul. 1990. [百度学术]
J. Beerten, S. Cole, and R. Belmans, “A sequential AC/DC power flow algorithm for networks containing multi-terminal VSC HVDC systems,” in Proceedings of IEEE PES General Meeting, Providence, USA, Sept. 2010, pp. 1-7. [百度学术]
W. Feng, C. Yuan, Q. Shi et al., “Using virtual buses and optimal multipliers to converge the sequential AC/DC power flow under high load cases,” Electric Power Systems Research, vol. 177, pp. 1-9, Dec. 2019. [百度学术]
J. Wu and A. Bose, “Parallel solution of large sparse matrix equations and parallel power flow,” IEEE Transactions on Power Systems, vol. 10, no. 3, pp. 1343-1349, Aug. 1995. [百度学术]
N. Garcia, “Parallel power flow solutions using a biconjugate gradient algorithm and a Newton method: a GPU-based approach,” in Proceedings of IEEE PES General Meeting, Providence, USA, Sept. 2010, pp. 1-4. [百度学术]
C. Cheng, L. Bin, J. Shen et al., “A modular parallelization framework for power flow transfer analysis of large-scale power systems,” Journal of Modern Power Systems and Clean Energy, vol. 6, pp. 679-690, Jul. 2018. [百度学术]
A. Lumsdaine, D. Gregor, and B. Hendrickson, “Challenges in parallel graph processing,” Parallel Processing Letters, vol. 17, no. 1, pp. 5-20, Mar. 2007. [百度学术]
T. Kajdanowics, P. Kazienko, and W. Indyk, “Parallel processing of large graphs,” Future Generation Computer Systems, vol. 32, pp. 324-337, Mar. 2014. [百度学术]
W. Feng, J. Wu, and C. Yuan, “A graph computation based sequential power flow calculation for large-scale AC/DC systems,” in Proceedings of 2019 IEEE PES General Meeting, Atlanta, USA, Aug. 2019, pp. 1-5. [百度学术]
A. Goldberg and C. Harrelson, “Computing the shortest path: a search meets graph theory,” in Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms, Vancouver, Canada, Jan. 2005, pp. 156-165. [百度学术]
Y. Lu, J. Cheng, D. Yan et al., “Large-scale distributed graph computing systems: an experimental evaluation,” Proceedings of the VLDB Endowment, vol. 8, pp. 281-292, Nov. 2014. [百度学术]
R. Chen, J. Shi, Y. Chen et al., “Powerlyra: differentiated graph computation and partitioning on skewed graphs,” Proceedings of the VLDB Endownement, vol. 5, Jan. 2019. [百度学术]
Q. Shi, C. Yuan, W. Feng et al., “Enabling model-based LTI for large-scale power system security monitoring and enhancement with graph-computing-based power flow calculation,” IEEE Access, vol. 7, pp. 167010-167018, Oct. 2019. [百度学术]
C. Yuan, Y. Zhou, G. Liu et al., “Graph computing-based WLS fast decoupled state estimation,” IEEE Transactions on Smart Grid, vol. 11, no. 3, pp. 2440-2451, May 2020. [百度学术]
C. Yuan, Y. Liu, W. Feng et al., “Graph computing based distributed fast decoupled power flow analysis,” in Proceedings of 2019 IEEE PES General Meeting, Atlanta, USA, Aug. 2019, pp. 1-5. [百度学术]
A. K. Srivastava, T. A. Ernster, R. Liu et al., “Graph-theoretic algorithms for cyber-physical vulnerability analysis of power grid with incomplete information,” Journal of Modern Power Systems and Clean Energy, vol. 6, pp. 887-899, Sept. 2018. [百度学术]
J. E. Gonzalez, Y. Low, H. Gu et al., “PowerGraph: distributed graph-parallel computation on natual graphs,” in Proceedings of 10th USENIX Symposium on Operating Systems Design and Implementation, Hollywood, USA, Oct. 2012, pp. 17-30. [百度学术]
A. Kyrola, G. Blelloch, and C. Guestrin, “GraphChi: large-scale graph computation on just a PC disk-based graph computation,” in Proceedings of 10th USENIX Symposium on Operating Systems Design and Implementation, Hollywood, USA, Oct. 2012, pp. 31-46. [百度学术]
R. Xin, J. E. Gonzalez, M. J. Fraklin et al., “GraphX: a resilient distributed graph system on Spark,” in Proceedings of First International Workshop on Graph Data Management Experience and Systems, New York, USA, Jun. 2013, pp. 1-6. [百度学术]
R. Dai, G. Liu, Z. Wang et al., “A novel graph-based energy management system,” IEEE Transactions on Smart Grid, vol. 11, no. 3, pp. 1845-1853, May 2020. [百度学术]
L. G. Valiant, “A bridging model for parallel computation,” Communications of the ACM, vol. 33, no. 8, pp. 103-111, Dec. 1990. [百度学术]
W. Feng, F. Li, C. Yuan et al., “Graph computation based power flow for large-scale AC/DC system,” in Proceedings of 2018 International Conference on Power System Technology (POWERCON), Guangzhou, China, Sept. 2018, pp. 468-473. [百度学术]
M. Adibi. (1993, Nov.). IEEE 300-bus test case. [Online]. Available: http://labs.ece.uw.edu/pstca/pf300/pg_tca300bus.htm [百度学术]
Texas A&M University. (2018, Aug.). South Carolina 500-bus system. [Online]. Available: https://electricgrids.engr.tamu.edu/electric-grid-test-cases/activsg500 [百度学术]