Journal of Modern Power Systems and Clean Energy

ISSN 2196-5625 CN 32-1884/TK

网刊加载中。。。

使用Chrome浏览器效果最佳,继续浏览,你可能不会看到最佳的展示效果,

确定继续浏览么?

复制成功,请在其他浏览器进行阅读

Scenario Generations for Renewable Energy Sources and Loads Based on Implicit Maximum Likelihood Estimations  PDF

  • Wenlong Liao
  • Birgitte Bak-Jensen
  • Jayakrishnan Radhakrishna Pillai
  • Zhe Yang
  • Yusen Wang
  • Kuangpu Liu
the AAU Energy, Aalborg University, Aalborg, Denmark; the School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden

Updated:2022-11-19

DOI:10.35833/MPCE.2022.000108

  • Full Text
  • Figs & Tabs
  • References
  • Authors
  • About
CITE
OUTLINE

Abstract

Scenario generations for renewable energy sources and loads play an important role in the stable operation and risk assessment of integrated energy systems. This paper proposes a deep generative network based method to model time-series curves, e.g., power generation curves and load curves, of renewable energy sources and loads based on implicit maximum likelihood estimations (IMLEs), which can generate realistic scenarios with similar patterns as real ones. After training the model, any number of new scenarios can be obtained by simply inputting Gaussian noises into the data generator of IMLEs. The proposed approach does not require any model assumptions or prior knowledge of the form in the likelihood function being made during the training process, which leads to stronger applicability than explicit density model based methods. The extensive experiments show that the IMLEs accurately capture the complex shapes, frequency-domain characteristics, probability distributions, and correlations of renewable energy sources and loads. Moreover, the proposed approach can be easily generalized to scenario generation tasks of various renewable energy sources and loads by fine-tuning parameters and structures.

I. Introduction

THE integrated energy system is a standard large-scale, non-linear, and dynamic control system with various flexible resources, such as electric vehicles, wind turbines, photovoltaic (PV) systems, and heat pumps [

1]. To coordinate these flexible resources in complex integrated energy systems, accurate modeling of time-series curves, e.g., power generation curves and load curves, for renewable energy sources and loads is necessary. One of the most popular methods to model time-series curves is scenario generation, which generate a large number of possible new time-series scenarios to represent uncertainties of the renewable energy sources and loads based on historical scenarios [2]. To some extent, the scenario generation can be considered as the first step of scenario forecasts, which trains a model to generate realistic scenarios as the test set, and then iteratively solves high-quality forecasting scenarios given a pre-trained model and deterministic point forecasting values. More details on why it makes sense to create new scenarios based on historical scenarios can be found in [3].

The core thinking of the scenario generation aims to produce a range of new time-series curves, which look like historical time-series curves. With regard to whether statistical hypotheses about probability distribution are needed, existing methods of scenario generations can be subsumed under just two categories: explicit density model based methods and implicit density model based methods. A brief comparison between the above two methods is shown in Table I. Specifically, the explicit density model based methods have to manually assume the probability density functions (PDFs), and then fit the unknown parameters in the PDFs by employing historical time-series curves. The widely-used explicit density model based methods involve the normal distribution model [

4], Gaussian mixture model [5], Copula theory based model [6], and autoregressive moving average (ARMA) model [7]. These explicit density model based methods have sound theoretical bases, and they are more interpretable than implicit density model based methods, which are regarded as black boxes. However, the quality of time-series curves generated from explicit density model based methods is poor, since the time-varying and dynamic properties of time-series curves make explicit density model based methods hard to scale and difficult to apply [8], particularly when multiple renewable power sources and loads are considered. In addition, these explicit density model based methods typically require statistical hypotheses of PDFs that may not hold in practice engineering [9]. Moreover, the PDFs of renewable energy sources and loads vary from region to region, and therefore the explicit density model based methods have difficulty in being generalized to other places [10]. Relatively, the explicit estimation of the PDFs of renewable energy sources and loads is not necessary for implicit density model based methods. Moreover, the implicit density model based methods can be easily generalized to scenario generation tasks of various renewable energy sources and loads in different regions by fine-tuning parameters and structures [11]. The popular implicit density model based methods mainly include generative adversarial networks (GANs) [12], hidden Markov models (HMMs) [13], non-linear independent component estimations (NICEs) [14], and variational auto-encoders (VAEs) [15]. Specifically, HMMs have sound theoretical bases and simple structures, but they have difficulty in capturing the spatio-temporal characteristics of time-series curves, since the assumption of independence of their output values leads to the lack of context information. Deep neural networks are powerful generative models [16], which are independently developed for scenario generation tasks. Nevertheless, the scale of parameters to be trained in NICEs is very large, which increases the difficulty in adjusting parameters. The performance of VAEs is relatively weaker, since they are not able to accurately calculate the lower bound of the log-likelihood of the historical time-series curves. The training problems of GANs, e.g., instability, non-convergence, and mode collapse, still exist in previous publications [17], [18]. In general, the previous implicit density model based methods involve either inaccurate loss functions or training problems, which seriously limit the quality of the new scenarios. Therefore, it is necessary to propose a new method with a stable training process, which can produce high-quality scenarios.

TABLE I  Comparison Between Explicit Density Model Based Methods and Implicit Density Model Based Methods
MethodModelAdvantageDisadvantageReference
Explicit density model based methods Normal distribution

1) Sound theoretical bases

2) Good model interpretability

1) Requirement for statistical hypotheses

2) Limited quality

3) Poor generalizability

[4]
Gaussian mixture model [5]
Copula theory based model [6]
ARMA [7]
Implicit density model based methods HMM

1) No statistical hypotheses

2) Strong generalization

Ignoration of spatio-temporal correlations of power generation curves [13]
NICE Too many hyper-parameters and limited quality of samples [14]
VAE Inability to accurately calculate lower bound of log-likelihood of historical curves [15]
GAN Unstable training process, e.g., instability, non-convergence, and mode collapse [16]-[18]

The implicit maximum likelihood estimation (IMLE) is a well-known deep generative network widely used in the computer vision field [

19]. Compared with other generative networks, e.g., VAEs, GANs, and NICEs, the quality of samples generated from IMLEs is better in many fields, and its training process is relatively more stable [20]. This is because its loss function ensures that the real examples have a generated sample nearby at optimality, but does not minimize the distinguishability between the datasets and new samples. In addition, gradients are not vanishing, since the gradients of the distances between real examples and their nearest samples don’t become zero. At present, IMLEs have shown outstanding performance in various fields such as image generation, style transfer, and missing data imputation [21], [22]. The successful applications of IMLEs in speech synthesis and image processing demonstrate that they can capture complicated patterns of high-dimensional samples. Therefore, the IMLE can be treated as a candidate for scenario generation. In theory, IMLEs can not only effectively extract latent representations of time-series curves for renewable energy sources and loads through convolutional neural networks (CNNs) with outstanding extracting performance, but also employ the data generator with stable loss function to generate high-quality new scenarios. However, most of existing IMLEs are designed for the high-dimensional image data with the same size of rows and columns, and these structures and parameters are not applicable for 1-D time-series curves of renewable energy sources and loads. Therefore, how to design a structure of IMLEs with strong performance according to the data characteristics of renewable energy sources and loads needs further study. In [23], an unsupervised deep learning framework using IMLEs is employed to generate scenarios for a single wind farm based on historical samples, but they ignore the spatiotemporal correlations between multiple adjacent wind farms. In this paper, the IMLEs are generalized to scenario generation of multiple correlated renewable energy sources, e.g., wind farms, PV plants, and loads including heating, cooling, and power loads in integrated energy systems. Compared with the work in [23], this paper focuses on designing the IMLEs to improve the quality of scenario generation accounting for correlations among the renewable energy sources and loads. Moreover, the performance and key parameters of the proposed approach are discussed by three different datasets. The key contributions are summarized as follows.

1) The IMLEs are generalized from scenario generation of a single wind farm into scenario generation of multiple correlated renewable energy sources and loads. The specific structure of IMLEs is designed to generate scenario for multiple units.

2) The proposed approach is completely based on data-driven generative networks, which project Gaussian noises into high-dimensional time-series curves. After unsupervised learning, IMLEs can directly generate realistic scenarios for PV plants, wind farms, heating, cooling, and power loads, which hold the same characteristics, e.g., correlations, frequency-domain characteristics, fluctuations, and probability distributions, as the real ones.

3) Different from explicit density model based methods, the proposed approach sidesteps manual assumption of the PDFs of renewable energy sources and loads, resulting in stronger generalization. By fine-tuning parameters and structures of the data generator, IMLEs can be easily used to produce new scenarios for multiple adjacent renewable energy sources and loads by simply inputting Gaussian noises to the pre-trained IMLEs. Besides, there is no limit to the number of new scenarios.

4) Compared with other implicit density model based methods (e.g., GAN), the proposed approach can sidestep unstable training problems, e.g., non-convergence and mode collapse, since the loss function ensures that each real example has a generated sample nearby at optimality, and the training process is a simple optimization problem that minimizes the loss function.

The rest of this paper is organized as follows. Section II formulates the scenario generation using IMLEs. Section III presents the case study. Section IV presents the conclusions and future works.

II. Scenario Generation Using IMLEs

A. Introduction of Scenario Generation

Generally, the scenario generation tasks can be divided into two categories: ① scenario generation for a single renewable energy resource or load; and ② scenario generation for multiple adjacent renewable energy resources or loads.

For the single renewable energy resource or load generation, training samples include a series of historical power generation curves or load curves X=[x1, x2, ..., xm], and the goal is to train a data generator based on IMLEs by using real samples from the training set. As shown in Fig. 1, the data generator can be considered as a mapping function which projects Gaussian noises into real samples, and the loss function is a kind of similarity metric between real and generated samples such as the Euclidean distance. After the unsupervised training process, Gaussian noises are used as inputs, and the data generator generates new samples, which can describe the same stochastic process as the real samples and exhibit a variety of different patterns. To prevent the model from simply replicating the samples from the training set, the performance of the data generator should be evaluated by comparing the similarity metric between real and generated samples from the test set rather than the training set.

Fig. 1  A simple structure of proposed approach.

In integrated energy systems, multiple adjacent renewable energy resources or correlated loads may need to be considered at the same time. Therefore, the scenario generation of a single time series can be generalized to the scenario generation of multiple time series. Specifically, historical samples {X1,X2,...,XN} of N renewable energy resources or loads are used to train a data generator whose goal is to simultaneously generate multiple scenarios. The generated samples should represent probability distributions of renewable energy resources or loads as well as the correlations. For scenario generation of renewable energy sources, the correlations include the spatial and temporal correlations of power generation curves. For the scenario generation of multiple loads, e.g., heating, cooling, and power loads, the correlations cover the temporal correlations of load curves and the correlations (also called coupling) between multi-class loads.

B. Loss Function of IMLEs

Normally, deep generative networks can be naturally viewed as a sampling procedure from a prior probability distribution, e.g., Gaussian distribution, and then noises are fed to the data generator to obtain the new samples [

23]. Gaussian distributions and data generators can be represented as:

z~N(0,1)x'=g(z) (1)

where z is a Gaussian noise; and x' is a new scenario generated from the data generator g() of IMLEs.

The probability distribution of the real scenario x is expressed as:

q(x)=q(z)q(x|z)dz (2)

where q(z) is the Gaussian distribution; q(x) is the probability distribution of the real scenario x; and q(x|z) is a conditional Dirac distribution or conditional Gaussian distribution.

Theoretically, (2) can fit any probability distribution, covering the probability distribution of real samples. Furthermore, suppose q(x|z) is a Dirac distribution. Equation (2) can be rewritten as:

q(x)=δ(x-g(z))q(z)dz=Ez~q(z)(δ(x-g(z))) (3)

where δ() is the Dirac function; and E() is the expected value.

Actually, the Dirac function can be regarded as a Gaussian distribution with the variance close to zero, and its mathematical formula is expressed as:

δ(x)=limσ01(2πσ2)d/2exp-x22σ2 (4)

where σ is the standard deviation; and d is the dimension of noises z.

Moreover, (4) is substituted into (3) to obtain the following results:

q(x)=limσ0Ez~q(z)1(2πσ2)d/2exp-x-g(z)22σ2 (5)

Suppose p(x) is the probability distribution of real scenarios. The training process of generative networks is to minimize the following loss function by the gradient descent method:

Ex~p(x)(lgq(x))=-p(x)lgq(x)dz (6)

By substituting (5) into (6), a more specific loss function L can be obtained as:

L=Ex~p(x)-lglimσ0Ez~q(z)1(2πσ2)d/2exp-x-g(z)22σ2 (7)

Obviously, there are few constants in (7) that will not affect the training results of IMLEs. In order to make the loss function more concise, these dispensable constants are ignored, and then a new form of the loss function is obtained as:

LEx~p(x)-lglimσ0Ez~q(z)exp-x-g(z)22σ2 (8)

In the training stage of IMLEs, n Gaussian noises Z=[z1,z2,,zn] and m real scenarios X=[x1,x2,,xm] are fed to IMLEs to the last form of the loss function [

19]:

L1mi=1mlgj=1nexp-xi-g(zj)22σ21mi=1mminj=1nxi-g(zj)2 (9)

In this case, this loss function can be used to update the weights of IMLEs by the back propagation algorithm.

C. Structure of IMLEs

The recent success of CNNs has boosted researches on the pattern recognition. Many data mining tasks such as target detection and speech recognition, which are once heavily dependent on artificial feature engineering to extract informative features, have been revolutionized by CNNs with powerful feature representing ability [

24]. Therefore, CNNs are employed to construct the data generator for scenario generations of renewable energy sources and loads.

For the scenario generation task, the CNNs mainly consist of transposed convolutional (TransConv) layers and dense layers. Specifically, the input features are performed with TransConv operations, and then a bias vector is added to obtain the output data of the TransConv layers. Its mathematical formula is expressed as:

Ytran=ftran(Xtran*Wtran+Btran) (10)

where Ytran is the output data of TransConv layers; Xtran is the input data of TransConv layers; ftran() is the activation function of TransConv layers; Wtran is the weight of TransConv layers; Btran is the bias vector of TransConv layers; and * is the TransConv operation.

Similarly, the output data of dense layers can be obtained by multiplying input data with weights and adding a bias vector. Its mathematical formula is expressed as:

Ydense=fdense(XdenseWdense+Bdense) (11)

where Ydense is the output data of dense layers; Xdense is the input data of dense layers; fdense() is the activation function of dense layers; Wdense is the weight of dense layers; and Bdense is the bias vector of dense layers.

D. Process of Scenario Generation

To summarize the above description, the steps of scenario generation for multiple renewable energy sources and loads based on IMLEs are introduced as follows.

1) Data Preprocess

Firstly, the datasets of renewable energy sources and loads are imported. 80% of the samples are randomly selected as the training set, and the remaining samples are used as the test set. Before inputting real time-series curves of renewable energy sources and loads into IMLEs, they should be normalized; otherwise, IMLEs are hard to converge in the training process. Hence, the minimum-maximum normalization method is employed to project input data to values from zero to one [

10].

It is widely known that IMLEs are originally used to produce images with the same size of columns and rows. Although the successful applications in image generation have proven that IMLEs can accurately capture complex characteristics of high-dimensional data through unsupervised learning, they have difficulty in directly processing the time-series curves where the size of rows and columns are different. Therefore, the time-series curves of renewable energy sources or loads should be reshaped into a square matrix before being input to IMLEs.

The rules of date transformation, i.e., converting time-series curves into matrices, are shown in Fig. 2. For a single power generation curve, the wind power generation curve of 1×144 scale is regarded as an example. The Reshape function from Python is employed to transform the original wind power curve into a square matrix of 12×12 scale, which can be input to IMLEs directly. In the same way, the time-series curves of renewable energy sources or loads can be reshaped into a square matrix for multiple power generation curves. For the time-series curves which cannot be directly reshaped into a square matrix, it is necessary to insert some zero elements into the end of time series [

10]. For example, the wind power generation curve is 1×48 scale, if the time resolution is 30 min. A zero element can be added to its end, and then it can be reshaped into a square matrix of 7×7 scale. In this case, the 2-D matrix reshaped from the original time-series curves can be fed directly to the data generator of IMLEs.

Fig. 2  Rules of data transformation. (a) A wind power curve to a 2-D matrix. (b) A PV power curve to a 2-D matrix. (c) A load curve to a 2-D matrix. (d) Multiple wind power curves to a 2-D matrix. (e) Multiple PV power curves to a 2-D matrix. (f) Multiple load curves to a 2-D matrix.

2) Initialize Structure

Before training IMLEs, some key parameters and structures such as choice of the optimizer, number of iterations, and number of hidden layers need to be initialized. Normally, the control variable method can be used to select these parameters [

25], which will be discussed in Section III-B.

3) Train IMLEs

To obtain a batch of noises Z=[z1,z2,,zn], the Monte Carlo method is employed to sample the Gaussian distribution. Then, these Gaussian noises are input into the data generator of IMLEs to generate the corresponding new scenarios X̂=[x̂1,x̂2,,x̂n]. Moreover, a batch of real scenarios is selected from the training samples randomly. The Euclidean distances between the real scenario xi and all new scenarios X̂ are calculated to find the closest new scenario x̂p(i).

To update the weights of data generator by the gradient descent method and back propagation method, the loss function 1mi=1mxi-x̂p(i)2 of IMLEs is calculated. When the number of iterations is exceeded, the pre-trained data generator will be output.

4) Evaluate Performance

After training IMLEs, batches of Gaussian noises are fed to the pre-trained data generator of IMLEs to obtain the corresponding new time-series curves. Furthermore, the new scenarios are also the matrices with the same size of columns and rows, and they need to be transformed into vectors through the inverse Reshape function. Last but not least, samples from the test set are used to evaluate the performance of the proposed approach.

III. Case Study

A. Data Description and Deep Learning Library

To fully test the performance of the proposed approach for scenario generations of renewable energy sources and loads, numerical simulations are performed on three real datasets. Specifically, the solar integration data and wind integration data were published by the National Renewable Energy Laboratory of the United States [

26], [27]. The nine adjacent PV plants and wind farms are used as examples to verify the effectiveness of the proposed approach. Both PV plants and wind farms include 1460 power generation curves with a time resolution of 10 min. A third dataset comes from the University of Texas at Austin [28], which includes 160 buildings, 70000 students, staff, and faculty. The heating, cooling, and electric supply are provided from the Hal C. Weaver power plant and its associated facilities. From July 2011 to September 2012, hourly heating, cooling, and power load data are recorded in this dataset.

In all datasets, 80% of the samples are randomly selected as the training set, and the remaining samples are used to evaluate the performance of the proposed approach. The programs are implemented in Spyder 3.2.8 with deep frameworks, including Tensorflow 1.0 and Keras 2.0.

B. Discussion of Key Parameters

The scenario generation of a single wind farm is utilized as an example to illustrate how to select the key parameters of IMLEs. Specifically, the basic units which are commonly used to construct the hidden layers in the field of deep learning include dense layers, TransConv layers, long short-term memory (LSTM) layers, and gated recurrent unit (GRU) layers. To find the suitable basic units for IMLEs, the loss functions of IMLEs consisting of different layers are calculated 20 times, and the average loss functions are given in Table II.

TABLE II  Average Loss Functions of IMLEs Consisting of Different Layers
No. of hidden layersAverage loss function of IMLEs
TransConvLSTMDenseGRU
1 66.63 67.69 69.51 69.85
2 65.32 68.30 67.89 67.82
3 63.55 68.94 68.14 69.81
4 63.64 72.72 67.14 69.00
5 64.36 69.30 67.51 74.23
6 64.42 67.67 65.85 70.75
7 67.26 69.28 66.03 67.98

The following conclusions can be drawn from Table II.

1) The LSTM layer based IMLEs and the GRU layer based IMLEs have similar performance, which are inferior to the Dense-based IMLEs. In contrast, the TransConv layer based IMLEs significantly outperform other IMLEs in most cases, since CNNs are better at extracting the latent features of the data.

2) With the increase of the number of hidden layers, the loss function of TransConv layer based IMLEs first decreases and then increases again, because multiple TransConv layers can improve the representation ability of IMLEs. However, too many hidden layers lead to an over-fitting problem. Generally, three TransConv layers can be considered as a good starting point for IMLEs.

As shown in Fig. 3, the suitable structures and parameters of IMLEs for scenario generation of PV plants and wind farms can be found based on the conclusions from Table II.

Fig. 3  Structures and parameters of IMLEs for scenario generations. (a) Scenario generations for a single generation unit. (b) Scenario generations for nine adjacent generation units.

Figure 3(a) is the structure of scenario generation for a single generation unit, such as a wind farm or PV plant, and Fig. 3(b) is the structure of scenario generation for nine adjacent generation units. Comparing the structures of these two sub-figures, it is found that IMLEs can be easily generalized to a variety of scenario generation problems (e.g., wind and PV power generation curves) by fine-tuning parameters.

Taking Fig. 3(a) as an example to explain parameters, the data generator mainly includes a dense layer and three TransConv layers. Specifically, the rectified linear unit (ReLU) function is the activation function of the dense layer with 228 neurons. There are 32 filters in the first TransConv layer and 16 filters in the second TransConv layer. The number of the filters in the last TransConv layer is 1. The hyperbolic tangent (Tanh) function is the activation function of the last TransConv layer, and the remaining TransConv layers use ReLU functions as their activation functions.

Note that the parameters in TransConv layers (e.g., filter, kernel, and stride) generally depend on experience and application (the shape of the output data) [

29]. Specifically, as the kernel size becomes larger, each feature is spread out from the input layer to a larger region. Therefore, the larger the kernel size is, the larger the output matrix is. The stride size indicates the speed at which the kernel moves through the output layer. The larger the stride size is, the larger the output matrix is. The filter represents the dimensionality of the output space. If a large amount of points are necessary for the network to generate the object, the generator may use bigger filters. If objects are somewhat small or have local features, the generator considers applying smaller filters. Last but not least, CNNs have been widely used for different tasks, so when designing the data generator, the parameters (e.g., filter, kernel, and stride) in TransConv layers can be borrowed from existing network structures of previous publications.

After initializing the structure of IMLEs, the gradient descent methods are applied to optimize the loss function of IMLEs. Furthermore, popular gradient descent methods include Adagrad, Adamax, SGD, Adam, Nadam, RMSProp, and AdaDelta [

30], which are used as black boxes in deep learning frameworks. To show how to choose an appropriate optimizer for IMLEs in scenario generation of wind farms, the loss functions of IMLEs with different optimizers are calculated 20 times, and the average loss functions are given in Table III.

TABLE III  Average Loss Functions of IMLEs with Different Optimizers
OptimizerLoss functionOptimizerLoss function
AdaDelta 238.29 Adamax 67.83
Adagrad 110.20 SGD 65.08
Adam 63.23 RMSProp 63.55
Nadam 63.52

IMLEs have good performances when Adamax, SGD, Nadam, RMSprop, and Adam are used as optimizers of IMLEs. In particular, the loss functions of Adam algorithm is the smallest, indicating that the Adam algorithm is the best choice for IMLEs. In addition, the loss functions of AdaDelta and Adagrad are significantly larger than those of other methods, which shows that they are not suitable for scenario generation via IMLEs.

In Fig. 4, the training evolution of the IMLEs on the wind dataset is shown to explore the training stability, convergence performance, and the minimum iteration of IMLEs.

Fig. 4  Training evolution of IMLEs on wind dataset.

Along with the increasing iteration times, the loss function of IMLEs decreases rapidly. When iteration times are more than 100, the loss function of IMLEs is getting close to a constant, which indicates that the data generator has converged. Unlike other generative networks (e.g., GAN) whose loss function involves instability or non-convergence [

8], the convergence speed of IMLEs is fast and the training process is stable relatively. In short, the number of iterations can be initialized to 100, which is sufficient to ensure the maturity of IMLEs. When the iteration procedure ends, the pre-trained IMLEs with outstanding performance are used to generate scenarios for renewable energy sources and loads.

C. Scenario Generation for Renewable Energy Sources

Normally, relevant works visualize a part of real images and their closest new images, which are the final result of the actual optimization, to verify the effectiveness of different methods in computer visions [

19]-[21]. By analogy, the mainstream publications analyze the performance of the generative models by comparing some real power generation curves from the test set with their closest new power generation curves [8]-[10]. Only comparing the real power generation curves with their closest new power generation curves cannot adequately illustrate the effectiveness of the proposed method, because some bad cases may be ignored. Further, the PDFs are employed to compare the overall similarity between all the new samples and real samples [15]. If the difference between new samples and real samples is very small, their PDFs would almost overlap. In contrast, the PDFs of new samples significantly vary from real ones, if there are a large number of unrealistic new samples. In short, this section verifies the effectiveness of the proposed method by comparing the similarity between generated samples and real samples from the test set, as previous publications do.

In order to check if real scenarios and new scenarios generated from IMLEs have similar properties, 2500 Gaussian noises are used as inputs of the data generator respectively, so as to produce 2500 PV power generation curves and 2500 wind power generation curves. Next, a portion of real scenarios with specific properties (e.g., sharp fluctuation, fast ramps, and large peaks) from the test set are picked to calculate Euclidean distances with the new scenarios. Finally, Fig. 5 visualizes the picked real scenarios and their nearest new scenarios when the iteration procedure ends.

Fig. 5  Visualization of real scenarios and their closest new scenarios generated from IMLEs. (a) Three samples of wind farms. (b) Three samples of PV plants.

The first row of Fig. 5 presents the shapes of new wind and PV power generation curves generated from IMLEs, which are similar to real power generation curves. It is difficult to distinguish real samples from new ones. The IMLEs are able to accurately capture the complex dynamic characteristics of power generation curves, such as sharp fluctuation, fast ramps, large peaks, and valleys. The real scenarios selected from the test set are not used to train IMLEs, while wind and PV power generation curves generated from IMLEs are the same as the shapes of real scenarios in the test set. This indicates that IMLEs have outstanding generalization ability rather than simply replicating the samples from the training set.

Aside from the shapes of power generation curves, it is necessary to verify some common statistical properties between real and new power generation curves generated from IMLEs.

For example, the temporal correlation is a typical property at the wind and PV power generation curves, which are often represented by the auto-correlation function [

17]. To check whether new and real power generation curves have similar temporal correlations, the auto-correlation functions are shown in the second row of Fig. 5. The auto-correlation functions of wind and PV power generation curves generated from IMLEs resemble those of the real scenarios closely, which show that IMLEs can capture the temporal correlations of the real power generation curves accurately.

Duration curves of wind farms and PV plants reflect the variation of output powers in a downward form [

10]. The area under the duration curves of renewable energy sources represents all energy produced by PV plants or wind farms in a day. Further, the third row of Fig. 5 shows the duration curves of PV plants and wind farms. It is found that the trends of power duration curves between the generated new scenarios and the real scenarios are basically the same, and the areas surrounded by the y-axis and x-axis are extremely similar, which shows that the new scenarios generated from IMLEs can reflect the total energy of real wind and PV power generation curves well.

The frequency-domain properties and fluctuations of power generation curves play an important role in the stable operation of integrated energy systems. To represent the energy value of frequency components in wind and PV power generation curves, the power spectral densities (PSDs) are used to evaluate the frequency-domain characteristics of power generation curves [

15].

From the fourth row of Fig. 5, it is clear that the trends and shapes of PSDs in the scenarios generated form IMLEs are consistent with the real scenarios, which indicates that the new scenarios generated from IMLEs can fit the fluctuation components of wind and PV power generation curves at different frequencies of the real scenarios well.

The Pearson correlation coefficient is one of the popular indices to evaluate the relationship between continuous variables, and it is often used to measure the linear relationship of wind and PV power generation curves at different look-ahead times. To further verify if new scenarios generated from IMLEs have similar temporal characteristics as the real scenarios, Fig. 6 visualizes the Pearson correlation matrices of real power generation curves and new power generation curves for wind farms and PV plants.

Fig. 6  Pearson correlation matrices of different scenarios. (a) Real wind power scenarios. (b) New wind power scenarios generated from IMLEs. (c) Real PV power scenarios. (d) New PV power scenarios generated from IMLEs.

The following conclusions can be drawn from Fig. 6.

1) For PV power generation curves, their time horizons of Pearson correlation matrices are different from those of wind power generation curves. This is because PV power is zero in the morning and at night, and Pearson correlation coefficient does not exist during these periods. Moreover, with the increase of time, the Pearson correlation coefficients between current wind power and previous wind power gradually decrease, while the temporal characteristics of PV power decrease first and then increase.

2) The Pearson correlation matrices of real scenarios and generated scenarios are very similar, which shows that without any statistical hypotheses of probability distributions being made during the training process, IMLEs can capture the temporal dependency of wind and PV power generation curves accurately.

Previous publications have shown that there are strong spatial characteristics between adjacent PV plants or wind farms [

8], [9], which play a significant role in the operation of integrated energy systems. Hence, the stochastic scenario generation of renewable energy sources needs to consider spatial correlation.

In order to qualitatively analyze the performance of IMLEs in capturing the spatial correlations of renewable energy sources, Fig. 7 shows the real power generation curves and new power generation curves generated from IMLEs of nine adjacent PV wind farms and nine adjacent power plants in different colors.

Fig. 7  Real power generation curves of nine adjacent renewable energies. (a) Nine real wind power scenarios. (b) Nine new wind power scenarios generated from IMLEs. (c) Nine real PV power scenarios. (d) Nine new PV power scenarios generated from IMLEs.

It is obvious that the real power generation curves of nine wind farms and nine PV plants have the same trend, which indicates that there are strong spatial correlations among these adjacent wind farms and PV plants. IMLEs are able to account for the spatial correlations among multiple generation units while generating the stochastic scenarios for renewable energy sources, which is in line with the actual situation. Moreover, the nine generated PV power generation curves are lightly closer to each other than the real ones, but this small difference is acceptable, since the spatial correlation among multiple generation units is related to the time-varying environmental factors, e.g., cloud and light intensity.

Furthermore, the mean Pearson correlation coefficients of all samples are employed to quantitatively calculate the spatial correlations among multiple generation units, as shown in Fig. 8.

Fig. 8  Pearson correlation matrix among nine renewable energies. (a) Nine real wind power scenarios. (b) Nine new wind power scenarios generated from IMLEs. (c) Nine real PV power scenarios. (d) Nine new PV power scenarios generated from IMLEs.

As can be observed from Fig. 8, although some parts of the Pearson correlation coefficient among generated samples are less than those of real samples, the error of the Pearson correlation coefficient does not exceed 0.1. This quantitatively shows that IMLEs can well capture the spatial correlation of adjacent power generation units.

In addition to the spatiotemporal correlations of power generation curves, their amplitudes are also important attributes. An explicit density model based method (e.g., Copula method [

6]) and other implicit density model-based methods (e.g., VAE [15] and GAN [17]) are used as baselines. Figure 9 shows the PDFs of real scenarios and new scenarios generated from IMLEs and baselines.

Fig. 9  PDFs of PV and wind power generation curves. (a) Wind power generation curves. (b) PV power generation curves.

Besides, Table IV presents the Euclidean distances of the PDFs between real scenarios and new scenarios quantitatively. Obviously, there is a small difference of PDFs between real scenarios and the new scenarios generated from IMLEs. Compared with other generative networks (e.g., Copula method, VAEs, and GANs), two PDFs of IMLEs are closer to those of real samples. This demonstrates the outstanding capability of IMLEs to produce new scenarios for wind and PV power generation curves with accurate distribution characteristics.

TABLE IV  Euclidean Distances of PDFs Between Real Scenarios and New Scenarios
ModelEuclidean distance (p.u.)
Wind powerPV power
Copula 0.0946 0.0247
GAN 0.0487 0.0302
VAE 0.0580 0.0339
IMLE 0.0163 0.0139

D. Scenario Generation for Multiple Loads

To ensure that IMLEs have outstanding performance for scenario generation of heating, cooling, and power load curves, as shown in Fig. 10, the appropriate parameters of the data generator are determined by using the control variable method in [

25].

Fig. 10  Structure and parameters of IMLEs for scenario generation of heating, cooling, and power load curves.

Obviously, the structures and parameters of IMLEs for scenario generation of heating, cooling, and power load curves can be obtained by fine-tuning the structures of IMLEs for renewable energy sources in Fig. 3.

To check whether real load curves and new load curves generated from IMLEs have similar properties, 2500 Gaussian noises are used as inputs of the data generator to produce 2500 samples. Each generated sample includes a heating load curve, a cooling curve, and a power load curve. Then, a part of real scenarios with complex dynamics characteristics (e.g., large peaks and valleys) from the test set are selected to calculate Euclidean distances with the new load curves. Finally, Fig. 11 visualizes the selected real load curves and their closest new load curves when the iteration procedure ends.

Fig. 11  Visualization of real load curves and their closest new load curves generated from IMLEs. (a) Sample 1. (b) Sample 2.

Obviously, the generated heating, cooling, and power load curves closely resemble the real ones from the test set, which are not used in the training process of IMLEs. Next, the second row of Fig. 11 shows that the real-generated pairs of heating, cooling, and power load curves have very similar auto-correlation function values. The third row of Fig. 11 shows the duration curves for heating, cooling, and power loads. The comparison results show that the generated and real duration curves are highly overlapping, i.e., the area enclosed with the x-axis and y-axis is approximately equal, which indicates that the total energy demand of the heating, cooling and power load curves generated from IMLEs during a day is in line with the real ones. Finally, it can be found from the fourth row of Fig. 11 that the heating, cooling, and power load curves generated from IMLEs are close to the real power load curve values in each frequency component, which indicates that the fluctuation characteristics of the power sequence generated from IMLEs are consistent with those of the original sequence.

In order to further verify whether the new heating, cooling, and power load curves generated from IMLEs have the similar temporal correlation as the real ones, Fig. 12 uses colormaps to visualize the Pearson correlation matrices of real and generated heating, cooling, and power load curves.

Fig. 12  Pearson correlation matrices of different scenarios. (a) Real heating load scenarios. (b) Real cooling load scenarios. (c) Real power load scenarios. (d) New heating load scenarios generated from IMLEs. (e) New cooling load scenarios generated from IMLEs. (f) New power load scenarios generated from IMLEs.

Although the Pearson correlation coefficients between previous and current loads decrease with time, they are always greater than 0.8, indicating that the heating, cooling, and power load curves are strongly time dependent. Similar element values in these Pearson correlation matrices show that the proposed scenario generation approach can capture the temporal correlations of heating, cooling, and power load curves accurately.

Previous publications have shown that there are strong correlations (also called coupling) among heating, cooling, and power loads [

28], which have important implications for the stable operation and risk assessment of integrated energy systems.

Therefore, the correlations among heating, cooling, and power load curves need to be considered when generating stochastic scenarios for them.

In particular, the two samples in the first row of Fig. 11 qualitatively show that the cooling and power loads are positively correlated, while they are negatively correlated with the heating loads. IMLEs can account for these correlations among multi-class loads when generating new heating, cooling, and power load curves.

Furthermore, the mean Pearson correlation coefficients of all samples are employed to quantitatively evaluate the correlations among heating, cooling, and power loads, as shown in Fig. 13.

Fig. 13  Mean Pearson correlation coefficients among heating, cooling, and power loads. (a) Real samples. (b) New samples generated from IMLEs.

Apparently, the mean Pearson coefficient matrix of the new load curves does not differ much from the real ones.

The maximum error of Pearson coefficient matrix between the real samples and new one is 0.02, indicating that IMLEs can capture the correlations among heating, cooling, and power loads well.

In addition to the above properties, Fig. 14 shows the PDFs of real and new heating, cooling, and power load curves generated from IMLEs and popular baselines (e.g., Copula method, VAEs, and GANs). Also, Table V presents the Euclidean distances of the PDFs between real historical and new cooling, heating, and power load curves quantitatively.

Fig. 14  PDFs of load curves. (a) Heating load. (b) Cooling load. (c) Power load.

TABLE V  Euclidean Distances of PDFs Between Real and New Cooling, Heating, and Power Load curves
ModelEuclidean distance (p.u.)
Heating loadCooling loadPower load
Copula 0.033 0.088 0.038
GAN 0.030 0.055 0.030
VAE 0.027 0.059 0.027
IMLE 0.015 0.026 0.014

The probability distributions of the load curves generated from the different deep generative networks are generally consistent with the real situation, indicating that these models can simulate the distribution characteristics of the real load curves well. In addition, since the PDFs of IMLEs are the closest to those of heating, cooling, and power loads, it shows that IMLE outperforms Copula method, GANs, and VAEs in terms of probability distributions.

IV. Conclusion

This paper proposes a novel data-driven approach to improve the quality of stochastic scenario generation for renewable energy sources and loads in integrated energy systems based on IMLEs. After performing simulation analysis on the real dataset, the following conclusions are obtained.

1) Compared with the dense layer, LSTM layer, and GRU layer, the TransConv layer is more suitable for the data generator of IMLEs. Normally, the number of TransConv layers should not be too small or too large, and three TransConv layers can be considered as a good starting point for different datasets. In addition, the Adam algorithm is more suitable as the optimization algorithm for IMLEs in scenario generation tasks than other algorithms.

2) Unlike previous implicit density model based methods (e.g., GANs) whose loss functions are difficult to converge, IMLEs converge very quickly and are relatively stable throughout the training process. In addition, IMLEs capture the probability distributions of renewable energy sources and loads more accurately than popular generative models, such as the Copula method, VAEs, and GANs.

3) Simulation results show that IMLEs are able to accurately capture the signature properties (e.g., fluctuations, large peaks, and fast ramps), frequency-domain characteristics, and temporal correlations of renewable energy sources and load curves in integrated energy systems. Besides, the energy consumption or energy generation of the generated samples is very similar to that of the real samples.

4) When simultaneously generating new stochastic scenarios for multiple adjacent renewable energy sources, IMLEs can consider the spatial correlations among them. Similarly, IMLEs also take good account of the correlations (also called coupling) among heating, cooling, and power loads.

For future works, the IMLEs may be extended into conditional scenario generations with specified properties (e.g., heavy loads).

References

1

W. Wang, S. Huang, G. Zhang et al., “Optimal operation of an integrated electricity-heat energy system considering flexible resources dispatch for renewable integration,” Journal of Modern Power Systems and Clean Energy, vol. 9, no. 4, pp. 699-710, Jul. 2021. [Baidu Scholar] 

2

Q. Zhao, W. Liao, S. Wang et al., “Robust voltage control considering uncertainties of renewable energies and loads via improved generative adversarial network,” Journal of Modern Power Systems and Clean Energy, vol. 8, no. 6, pp. 1104-1114, Nov. 2020. [Baidu Scholar] 

3

Y. Wang, G. Hug, Z. Liu et al., “Modeling load forecast uncertainty using generative adversarial networks,” Electric Power Systems Research, vol. 189, pp. 1-9, Dec. 2020. [Baidu Scholar] 

4

J. Cao, W. Du, H. Wang et al., “Probabilistic load flow using Latin hypercube sampling with dependence for distribution networks,” in Proceedings of 2nd IEEE PES International Conference and Exhibition on Innovative Smart Grid Technologies, Manchester, UK, Dec. 2016, pp. 1-6. [Baidu Scholar] 

5

Z. Wang, C. Shen, F. Liu et al., “Analytical expressions for joint distributions in probabilistic load flow,” IEEE Transactions on Power Systems, vol. 32, no. 3, pp. 2473-2474, May 2017. [Baidu Scholar] 

6

Y. Qiu, Q. Li, Y. Pan et al., “Analytical expressions for joint distributions in probabilistic load flow,” International Journal of Hydrogen Energy, vol. 44, no. 11, pp. 5162-5170, Feb. 2019. [Baidu Scholar] 

7

Q. Tu, S. Miao, F. Yao et al., “Forecasting scenario generation for multiple wind farms considering time-series characteristics and spatial-temporal correlation,” Journal of Modern Power Systems and Clean Energy, vol. 9, no. 4, pp. 837-848, Jul. 2021. [Baidu Scholar] 

8

Y. Chen, Y. Wang, D. Kirschen et al., “Model-free renewable scenario generation using generative adversarial networks,” IEEE Transactions on Power Systems, vol. 33, no. 3, pp. 3265-3275, May 2018. [Baidu Scholar] 

9

Y. Chen, P. Li, and B. Zhang, “Bayesian renewables scenario generation via deep generative networks,” in Proceedings of 52nd Annual Conference on Information Sciences and Systems (CISS), Princeton, USA, Mar. 2018, pp. 1-6. [Baidu Scholar] 

10

L. Ge, W. Liao, S. Wang et al., “Modeling daily load profiles of distribution network for scenario generation using flow-based generative network,” IEEE Access, vol. 8, pp. 77587-77597, Apr. 2020. [Baidu Scholar] 

11

J. Li, J. Zhou, and B. Chen, “Review of wind power scenario generation methods for optimal operation of renewable energy systems,” Applied Energy, vol. 280, pp. 1-12, Dec. 2020. [Baidu Scholar] 

12

C. Wang, C. Xu, X. Yao et al., “Evolutionary generative adversarial networks,” IEEE Transactions on Evolutionary Computation, vol. 23, no. 6, pp. 921-934, Dec. 2019. [Baidu Scholar] 

13

Y Khalifa, D Mandic, and E Sejdi, “A review of hidden Markov models and recurrent neural networks for event detection and localization in biomedical signals,” Information Fusion, vol. 69, pp. 52-72, May 2021. [Baidu Scholar] 

14

L. Zhang and B. Zhang, “Scenario forecasting of residential load profiles,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 1, pp. 84-95, Jan. 2020. [Baidu Scholar] 

15

Z. Pan, J. Wang, W. Liao et al., “Data-driven EV load profiles generation using a variational auto-encoder,” Energies, vol. 12, no. 5, pp. 1-15, Mar. 2019. [Baidu Scholar] 

16

N. Hajarolasvadi, M. A. Ramírez, W. Beccaro et al., “Generative adversarial networks in human emotion synthesis: a review,” IEEE Access, vol. 8, pp. 218499-218529, Dec. 2020. [Baidu Scholar] 

17

J. Liang and W. Tang, “Sequence generative adversarial networks for wind power scenario generation,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 1, pp. 110-118, Jan. 2020. [Baidu Scholar] 

18

J. Qiao, T. Pu, and X. Wang, “Renewable scenario generation using controllable generative adversarial networks with transparent latent space,” CSEE Journal of Power and Energy Systems, vol. 7, no. 1, pp. 66-77, Jan. 2021. [Baidu Scholar] 

19

K. Li and J. Malik. (2018, Sept.). Implicit maximum likelihood estimation. [Online]. Available: https://arxiv.org/abs/1809.09087 [Baidu Scholar] 

20

Y. Hoshen, K. Li, and J. Malik, “Non-adversarial image synthesis with generative latent nearest neighbors,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, Jun. 2019, pp. 5804-5812. [Baidu Scholar] 

21

K. Li, T. Zhang, and J. Malik, “Diverse image synthesis from semantic layouts via conditional IMLE,” in Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Nov. 2019, pp. 4219-4228. [Baidu Scholar] 

22

S. Ravuri, S. Mohamed, M. Rosca et al., “Learning implicit generative models with the method of learned moments,” in Proceedings of 35th International Conference on Machine Learning, Hanoi, Vietnam, Sept. 2018, pp. 4314-4323. [Baidu Scholar] 

23

W. Liao, B. Jensen, J. Pillai et al., “Data-driven scenarios generation for wind power profiles using implicit maximum likelihood estimation,” in Proceedings of 12th International Conference on Applied Energy (ICAE2020), Bangkok, Thailand, Dec. 2020, pp. 1-5. [Baidu Scholar] 

24

W. Liao, B. Bak-Jensen, J. R. Pillai et al., “A review of graph neural networks and their applications in power systems,” Journal of Modern Power Systems and Clean Energy, vol. 10, no. 2, pp. 345-360, Mar. 2022. [Baidu Scholar] 

25

W. Liao, D. Yang, Y. Wang et al., “Fault diagnosis of power transformers using graph convolutional network,” CSEE Journal of Power and Energy Systems, vol. 7, no. 2, pp. 241-249, Mar. 2021. [Baidu Scholar] 

26

C. Draxl, A. Clifton, B. Hodge et al., “The wind integration national dataset (WIND) toolkit,” Applied Energy, vol. 151, pp. 355-366, Aug. 2015. [Baidu Scholar] 

27

National Renewable Energy Laboratory. (2012, Nov.). Solar integration national dataset toolkit. [Online]. Available: https://www. nrel. gov/grid/sind-toolkit.html [Baidu Scholar] 

28

K. Powell, A. Sriprasad, W. Cole et al., “Heating, cooling, and electrical load forecasting for a large-scale district energy system,” Energy, vol. 74, pp. 877-885, Sept. 2014. [Baidu Scholar] 

29

N. Aloysius and M. Geetha, “A review on deep convolutional neural networks,” in Proceedings of 2017 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, Apr. 2017, pp. 588-592. [Baidu Scholar] 

30

S. Ruder. (2016, Sept.). An overview of gradient descent optimization algorithms. [Online]. Available: https://arxiv.org/abs/1609.04747 [Baidu Scholar]