Abstract
The variable nature of solar power has hitherto been regarded as a major barrier preventing large-scale high-penetration solar energy into the power grid. Based on decades of research, particularly those advances made over the recent few years, it is now believed that dispatchable solar power is no longer a conception but will soon become techno-economically feasible. The policy-driven information exchange among the weather centers, grid operators, and photovoltaic plant owners is the key to realizing dispatchable solar power. In this paper, a five-step forecasting framework for enabling dispatchable solar power is introduced. Among the five steps, the first three, namely numerical weather prediction (NWP), forecast post-processing, and irradiance-to-power conversion, have long been familiar to most. The last two steps, namely hierarchical reconciliation and firm forecasting, are quite recent conceptions, which have yet to raise widespread awareness. The proposed framework is demonstrated through a case study on achieving effectively dispatchable solar power generation at plant and substation levels.
FORECASTING plays a major role during the integration of solar power, in particular, photovoltaic (PV) power. However, despite decades of research, the current accuracy level of solar power forecasting is still far from being comparable to that of load forecasting [
Indeed, according to the recent surveys in [
Regardless, firm forecast enablers have to work in line with the current grid integration policies. As such, the current solar forecasting practices are first reviewed in Section II-A. In a nutshell, the current forecasting practices are often associated with a two-way information flow, where national agencies issue weather forecasts to plant owners, who are responsible for post-processing and converting them to PV power forecasts, and submitting the PV power forecasts to the grid operators [
Insofar as solar forecasting for grid integration is concerned, there are two go-to methods, namely satellite-based forecasting and numerical weather prediction (NWP), with the former supporting intra-day forecasting and the latter supporting day-ahead forecasting [
That said, even forecasts from the best satellite- and NWP-based models are found to be biased and under-dispersed, due to our incomplete understanding of various atmospheric processes and lack of timely observations for data assimilation [
The third step involves converting the post-processed irradiance forecasts to PV power, which is commonly done via a model chain, or more formally, a solar power curve [
The three-step PV power forecasting framework, first of NWP, second of post-processing, and third of irradiance-to-power conversion, is familiar to many, and is generally regarded as the most promising and advanced one to date [
Many time series in social and physical settings, including the PV generation time series, can be framed into hierarchical structures. More specifically, the plant-level PV generation can be gathered according to distribution feeder nodes, balancing authority areas, load zones, or substations of different voltage levels. To give perspective,

Fig. 1 Locations and sizes of 405 simulated PV plants, 115-345 kV transmission lines, and 500 kV lines/substations in California, USA. (a) PV plants where colors are simply a visual aid to help distinguish overlapped dots. (b) High-voltage transmission lines and substations.
Whereas the actual measurements across a hierarchy sum up naturally, the forecasts do not, owing to the modeling uncertainty and information asymmetry. In a solar forecasting context, upper-level forecasts are usually produced by grid operators by extrapolating the aggregated historical generation time series outwards, whereas lower-level forecasts are produced, for example, with the aforementioned three-step framework. It would be very difficult to imagine that the lower-level forecasts, against all odds, sum up exactly to the higher-level ones, unless very naive methods such as persistence is used. On this point, hierarchical forecasting seeks to produce a set of aggregate consistent forecasts across the hierarchy, facilitating joint decision-making. Besides that, forecast reconciliation brings to the table several other benefits, such as accuracy improvements [
For deterministic forecast reconciliation, one of the best methods is known as the optimal reconciliation technique, which is able to minimize the reconciliation errors in the least-squares sense. Originally proposed in [
Denoting observation (i.e., PV power measurement) by , one can collect all observations made at time into a vector yt:
(1) |
where is the total PV power measured in the power system; and are the vectors of measurements corresponding to different levels of the hierarchy, with denoting the bottom level. According to the hierarchical nature,
(2) |
where S is a summing matrix. That said, can be constructed by the bottom-level measurements and , which contains only zeros and ones. To give perspective on the notation, a two-level hierarchy is used as an example in Supplementary Material A.
Similar to observation, the forecast is denoted by or x, and one may collect forecasts across the hierarchy at time into a vector :
(3) |
Aggregation inconsistency suggests:
(4) |
Denoting the reconciled version of forecasts by adding a tilde to the relevant expression, a set of aggregate consistent forecasts may be written as :
(5) |
In this regard, the goal of reconciliation is to find an optimal choice of matrix such that:
(6) |
, as in (3), is the vector of forecasts before reconciliation, which is otherwise known as the (incoherent) base forecast.
Numerous methods exist for obtaining . For instance, in the bottom-up (BU) method, which assumes that the higher-level forecasts are simply obtained by summing up lower-level ones, the matrix becomes :
(7) |
where is the total number of series in the hierarchy; is the number of bottom-level series in the hierarchy; is an identity matrix with the size of ; and is an matrix of zeros. Moving beyond the trivial BU reconciliation, the common problem-solving philosophy of statisticians is to find a choice of such that it minimizes some loss functions. On this point, the generalized least squares (GLS) reconciliation, which minimizes the reconciliation error in the least squares sense, becomes a natural choice.
(8) |
(9) |
is the conditional expectation of the bottom-level observation random vector at given previous observations up to time , i.e., . This idea of GLS reconciliation was first given in [
(10) |
where is the variance-covariance matrix of the -step-ahead reconciliation error , and is its Moore-Penrose generalized inverse. Consequently, substituting (10) into (6) yields (10), where under GLS reconciliation takes the form of given as:
(11) |
is unknown and can be hard to estimate, and thus alternative methods are needed. On this account, the so-called minimum-trace (MinT) reconciliation proposed in [
(12) |
where is the variance-covariance matrix of base forecast error :
(13) |
As such, the reconciled forecasts can be obtained from:
(14) |
where denotes under MinT reconciliation.
It should be noted that the MinT reconciliation minimizes the trace of the variance-covariance matrix of the reconciled forecast error:
(15) |
However, it only leverages the variance-covariance matrix of base forecast error, which can be seen in (13); this has been the foremost significance. To estimate , one has to shrink the entries of the -sample estimates of the variance-covariance matrix of the one-step-ahead forecast errors toward its diagonal:
(16) |
(17) |
where is the new version of ; is the base forecast error; is the in-sample estimate of the variance-covariance matrix of the one-step-ahead base forecast errors; is the diagonal matrix of ; and is a shrinkage parameter, which can be computed using the method of [
(18) |
This completes the optimal hierarchical reconciliation step.
Recall that the current grid integration adopts a two-way information flow, in that NWP-based forecasts are disseminated from the weather centers, and individual plant owners submit their converted power forecasts to grid operators [
To firmly meet the generation target (i.e., reconciled forecasts), one seeks to find an optimal mix of firm forecast enablers that is most economic. As geographical smoothing is passive, firm forecasting in this paper is modeled with two enablers, namely battery storage as well as overbuilding and proactive curtailment, without loss of generality. The logic flow is shown in

Fig. 2 Logic flow of dynamic curtailing strategy.
The objective function for the optimization problem is the firm forecast premium, which is denoted as and defined as the ratio between the levelized cost of electricity (LCOE) of firm forecasting and that of unconstrained PV:
(19) |
where denotes the LCOE of firm forecasting; and denotes the LOCE of unconstrained PV. LCOE is calculated as the ratio of the equivalent annual cost of a generation option over the equivalent annual electrical energy produced by that option. As such, can be written as the ratio of the equivalent annual cost of firm forecasting and annual PV forecast, whereas can be written as the ratio of the equivalent annual cost of unconstrained PV and annual electricity produced by unconstrained PV. The term “unconstrained PV” refers to the as-available (i.e., intermittent) PV installation [
The equivalent annual cost of firm forecasting comprises four parts: the investment cost of PV, the operation and maintenance (O&M) cost of PV, the investment cost of battery storage, and the O&M cost of battery storage. Mathematically, one can write the equivalent annual cost as:
(20) |
where and are the capital recovery factors of solar and battery, respectively; and are the O&M cost factors of solar and battery, respectively; and are the investment costs, which is in $/kW for PV and $/kWh for battery, respectively; is the rated power of unconstrained PV; is the rated capacity of the battery; is the overbuilding ratio of the PV; is the charging power of the battery at time , which differs from hour to hour. , , , , , , and are known; while , , and are to be determined by optimization. The capital recovery factors take the well-known form:
(21) |
where and are the lifetime of solar and battery, respectively; and is the discount rate.
The minimization problem in (20) must be subject to a series of constraints pertaining to the forecasts, operations of PV, and operations of battery.
PV power forecast made at time must be fulfilled by grid-injecting PV power at time and (if any) the discharging power at time :
(23) |
The power output of PV at any time must be injected into the grid, curtailed (if needed, denoted by ), and/or fed into the battery storage (if needed, denoted by ):
(24) |
If the battery is charged or discharged, the charging/discharging power must be within the limits:
(25) |
where and are the maximum charging and discharging power of the battery, respectively; and and are the binary variables representing the charging and discharging states of the battery, respectively, which must satisfy (27), assuming that the battery cannot be charged and discharged at the same time.
(27) |
For two consecutive hours and , the available energy in kWh of the battery must follow some continuity, in that, the available energy at time approximately equals the available energy at time plus (or minus) the charging (or discharging) power:
(28) |
where is the battery self-discharging rate; is the battery charging/discharging efficiency; and is the unit time interval.
The available energy of the battery cannot exceed at any time its rated capacity or fall below zero, i.e.,
(29) |
Besides , , and , which have appeared in the objective function, other variables that need to be optimized include , , , , , and . As and are binary integers, the minimization problem in (20) is a mixed-integer linear programming (MILP), for which standard solvers exist. To perform the optimization, non-variable parameter values should be first sought, typically from the literature. In this paper, the values are selected as [
This paper considers the Solar Power Data for Integration Studies (SPDIS), which is the data product of a three-phase project namely the Western Wind and Solar Integration Study, led by the National Renewable Energy Laboratory in USA. The SPDIS dataset contains two kinds of files, PV power generation (i.e., simulated “actuals”) and their corresponding forecasts. The PV power is simulated via the sub-hour algorithm in [
As the first step, the 5 min actual PV power data are aggregated to an hourly resolution to match the resolution of the 3Tier forecast. Next, the PV systems are framed into a two-level hierarchy based on the 500 kV substations. The PV-to-substation assignment is done with the nearest neighbor method, in which each system is allocated as a bottom-level node for the substation nearest to it.

Fig. 3 Assignment of 405 simulated PV plants to their nearest substations forming a two-level hierarchy.
No. | Longitude () | Latitude () | Name | Number of PV systems | Capacity of PV systems (MW) |
---|---|---|---|---|---|
1 | 121.56 | 37.71 | Tesla | 7 | 171 |
2 | 121.64 | 39.56 | Table Mt. | 3 | 136 |
3 | 120.70 | 36.72 | Oxford | 2 | 225 |
4 | 120.13 | 36.14 | Gates | 8 | 759 |
5 | 115.72 | 32.72 | Imperial Valley | 12 | 965 |
6 | 119.45 | 35.40 | Midway | 13 | 813 |
7 | 120.50 | 40.96 | Madeline | 1 | 7 |
8 | 118.48 | 34.28 | Rinaldi | 6 | 630 |
9 | 118.39 | 34.24 | Sta. M (Valley) | 18 | 1866 |
10 | 118.49 | 34.31 | Sylmar | 1 | 121 |
11 | 118.30 | 34.69 | Antelope | 24 | 1429 |
12 | 118.58 | 34.44 | Pardee | 12 | 650 |
13 | 118.12 | 34.49 | Vincent | 14 | 966 |
14 | 121.02 | 37.05 | Los Banos | 3 | 68 |
15 | 117.79 | 33.83 | Serrano | 30 | 2564 |
16 | 116.58 | 33.94 | Devers | 22 | 1290 |
17 | 117.16 | 33.74 | Valley | 24 | 1005 |
18 | 117.32 | 34.56 | Victorville | 5 | 652 |
19 | 115.52 | 35.25 | Cima | 14 | 1928 |
20 | 117.37 | 34.37 | Lugo | 3 | 306 |
21 | 117.56 | 34.01 | Mira Loma | 10 | 482 |
22 | 116.98 | 32.68 | Miguel | 30 | 451 |
23 | 121.94 | 40.81 | Round Mt. | 1 | 11 |
24 | 121.92 | 38.40 | VD&G Yard | 12 | 224 |
25 | 122.38 | 40.38 | Olinda | 3 | 112 |
26 | 117.44 | 34.55 | Adelanto | 20 | 1126 |
27 | 121.72 | 39.02 | O’Banion | 14 | 332 |
28 | 117.53 | 34.09 | Rancho Vista | 10 | 465 |
29 | 121.58 | 37.80 | Tracy | 16 | 378 |
30 | 121.78 | 36.81 | Duke Energy | 6 | 36 |
31 | 121.92 | 38.41 | Calpeak Power | 4 | 64 |
32 | 121.75 | 37.23 | Metcalf 2 | 27 | 531 |
33 | 120.85 | 35.22 | Diablo Canyon | 12 | 169 |
34 | 115.56 | 33.67 | Capacitor | 18 | 1571 |
The 3Tier forecasts are available for each PV plant. These are bottom-level (or ) base forecasts. To generate the upper-level forecasts, a time series method is used. Knowing that PV power exhibits diurnal transient, the autoregressive integrated moving average (ARIMA) model with Fourier terms (AFT) is thought adequate. AFT first fits a Fourier series to the PV power data, and models the remainder with ARIMA. AFT is applied for each top-level (or ) and middle-level (or ) series, separately. For each series, forecasts are produced in a rolling manner: data from January 1-7 are used to train the first AFT model, and forecasts for January 8 are produced; then data from January 2-8 are used to train the second AFT model, and forecasts for January 9 are produced. This continues until the forecasts for December 31 are produced.
With the base forecasts for all levels being ready, two reconciliation methods are used to produce reconciled forecasts, namely MinT (mentioned in Section II-B) and BU reconciliation, which simply sums up the forecasts to form aggregate consistent and forecasts. Whereas BU reconciliation does not require training, MinT reconciliation does. The one-year dataset is split into two halves chronologically. MinT is first trained with data from the first half of year, and out-of-sample reconciled forecasts are produced for the second half of year. Then, MinT is trained with data from the second half of year, and out-of-sample reconciled forecasts are produced for the first half of year. As such, one full year of MinT-reconciled forecasts is obtained.
In this subsection, firm forecasting is conducted at the substation level (i.e., ). Several important quantities pertaining to firm forecasting are first explained. The firm forecast premium, which is an overall indicator of the cost-effectiveness of firming up variable solar power generation, can be interpreted as the cost multiplier for meeting the forecast amount of PV power with absolute certainty. Firm forecast premium may be visualized as a function of the overbuilding ratio, which is the capacity multiplier to the original installed PV capacity, e.g., an overbuilding ratio of 2 means expanding whatever original installed capacity by two times. Battery storage required to firm up uncertain forecasts can be gauged in its usual unit, i.e., MWh or kWh. Last but not least, it is of interest to quantify the additional cost in $/kW, which indicates how much additional monetary investment is necessary to firm up a unit PV capacity. This quantity is called $/kW premium. All the above-mentioned quantities have a negative orientation, i.e., the lower their values are, the more amenable the strategy is.
To give perspective on interpreting the firm forecasting results,

Fig. 4 Firm forecast premium against overbuilding ratio for substation 8. (a) BU. (b) MinT.
The figure displays how battery (orange line) and PV (blue line) contribute to the overall firm forecast premium (green line), i.e., the green line is the sum of the orange and blue lines. Several points are marked in the figure, with the first number in each pair of parentheses being the overbuilding ratio and the second being the firm forecast premium. Point A corresponds to the situation with unconstrained PV, which in itself, though inexpensive, is not firm. To firm up the unconstrained PV with a storage-only solution (i.e., no overbuilding), an exceedingly high capacity of battery is needed, which in turn drives up the premium substantially (26.75 for BU or 13.82 for MinT). This situation is marked as point B. At point C, the firm forecasting strategy is optimal, which corresponds to the combination of battery and overbuilding that gives the lowest premium (2.46 for BU or 2.04 for MinT).
Then, as the overbuilding ratio continues to increase, the firm forecast premium is dominated by PV cost, and thus starts to rise linearly. Point D shows the premium at an overbuilding ratio of 3 (3.25 for BU or 3.15 for MinT), which is higher than the optimal premium.

Fig. 5 Substation-level RMSE versus $/kW premium using two reconciliation methods. (a) BU. (b) MinT.
Index | Capacity (MW) | RMSE (%) | Firm premium | Overbuilding ratio | Battery capacity (MWh) | Premium ($/kW) | |||||
---|---|---|---|---|---|---|---|---|---|---|---|
BU | MinT | BU | MinT | BU | MinT | BU | MinT | BU | MinT | ||
1 | 171 | 45.7 | 28.5 | 2.79 | 2.18 | 2.10 | 1.61 | 802 | 547 | 170 | 104 |
2 | 136 | 49.0 | 31.9 | 3.54 | 2.48 | 3.03 | 1.86 | 753 | 453 | 264 | 129 |
3 | 225 | 39.9 | 28.8 | 2.30 | 2.31 | 1.34 | 1.55 | 978 | 867 | 101 | 110 |
4 | 759 | 26.5 | 22.4 | 1.91 | 2.05 | 1.43 | 1.51 | 1651 | 2077 | 73 | 88 |
5 | 965 | 19.4 | 15.4 | 1.66 | 1.76 | 1.36 | 1.29 | 1781 | 2358 | 62 | 65 |
6 | 813 | 24.7 | 23.0 | 2.08 | 2.11 | 1.51 | 1.36 | 2676 | 3154 | 97 | 94 |
7 | 7 | 45.2 | 34.9 | 2.91 | 2.35 | 1.65 | 1.15 | 57 | 45 | 189 | 120 |
8 | 630 | 32.4 | 26.3 | 2.46 | 2.04 | 1.42 | 1.27 | 3667 | 2550 | 130 | 90 |
9 | 1866 | 30.7 | 24.7 | 2.09 | 1.88 | 1.67 | 1.41 | 4935 | 4621 | 100 | 76 |
10 | 121 | 36.0 | 29.6 | 2.72 | 2.31 | 1.47 | 1.22 | 855 | 698 | 155 | 114 |
11 | 1429 | 23.2 | 19.4 | 1.86 | 1.89 | 1.33 | 1.13 | 4636 | 5653 | 82 | 77 |
12 | 650 | 29.1 | 24.0 | 2.69 | 2.27 | 1.46 | 1.26 | 4670 | 3477 | 156 | 110 |
13 | 966 | 23.5 | 19.3 | 1.92 | 1.90 | 1.39 | 1.30 | 3120 | 3076 | 86 | 78 |
14 | 68 | 43.8 | 28.0 | 4.10 | 2.79 | 1.31 | 1.10 | 1031 | 595 | 271 | 151 |
15 | 2564 | 37.1 | 28.4 | 2.68 | 2.61 | 2.67 | 2.22 | 2995 | 5817 | 161 | 140 |
16 | 1290 | 21.2 | 17.0 | 1.76 | 1.74 | 1.41 | 1.28 | 2963 | 3049 | 73 | 63 |
17 | 1005 | 29.3 | 23.4 | 2.21 | 2.34 | 1.24 | 1.20 | 5337 | 5886 | 108 | 113 |
18 | 652 | 22.1 | 18.7 | 1.91 | 2.09 | 1.44 | 1.33 | 1893 | 2625 | 85 | 93 |
19 | 1928 | 22.2 | 18.2 | 1.83 | 1.77 | 1.40 | 1.30 | 4968 | 4732 | 77 | 66 |
20 | 306 | 31.0 | 24.8 | 2.36 | 2.42 | 1.32 | 1.65 | 1781 | 1229 | 123 | 120 |
21 | 482 | 33.1 | 26.4 | 2.09 | 2.10 | 1.47 | 1.31 | 1820 | 1963 | 102 | 94 |
22 | 451 | 36.6 | 27.6 | 2.60 | 2.56 | 2.14 | 1.81 | 1376 | 1809 | 147 | 134 |
23 | 11 | 56.5 | 41.5 | 3.73 | 3.45 | 2.38 | 1.68 | 86 | 99 | 246 | 207 |
24 | 224 | 50.2 | 37.1 | 2.51 | 2.66 | 1.46 | 1.93 | 1714 | 975 | 167 | 150 |
25 | 112 | 51.8 | 40.3 | 2.99 | 2.68 | 1.78 | 1.58 | 899 | 654 | 198 | 146 |
26 | 1126 | 23.0 | 18.6 | 1.84 | 1.96 | 1.21 | 1.20 | 4169 | 4494 | 80 | 83 |
27 | 332 | 53.5 | 27.4 | 3.89 | 2.36 | 2.82 | 1.69 | 3394 | 1220 | 321 | 119 |
28 | 465 | 30.2 | 25.2 | 2.15 | 2.17 | 1.31 | 1.12 | 2274 | 2498 | 107 | 99 |
29 | 378 | 41.4 | 28.1 | 2.73 | 2.20 | 1.35 | 1.72 | 3262 | 1030 | 171 | 106 |
30 | 36 | 50.2 | 35.4 | 2.51 | 2.16 | 2.21 | 1.63 | 91 | 105 | 145 | 102 |
31 | 64 | 50.7 | 29.4 | 3.60 | 2.34 | 2.94 | 1.97 | 450 | 137 | 279 | 118 |
32 | 531 | 44.6 | 31.6 | 2.29 | 2.16 | 1.61 | 1.65 | 2176 | 1538 | 120 | 103 |
33 | 169 | 34.5 | 25.3 | 2.11 | 2.50 | 1.50 | 1.18 | 557 | 1165 | 97 | 128 |
34 | 1571 | 19.4 | 16.4 | 1.85 | 2.09 | 1.47 | 1.38 | 3468 | 5797 | 76 | 92 |
Overall | 22503 | 28.9 | 22.7 | 2.29 | 2.12 | 1.62 | 1.45 | 77284 | 76993 | 109 | 95 |
Nonetheless, it is also true that more accurate forecasts do not necessarily lead to more economic firm forecasting. Reference [
Different from the previous subsection, this subsection investigates firm forecasting that is conducted at the individual plant level (i.e., ). This option of firm forecasting is reasonable, because the power grid may not want to be the sole bearer of the cost for attaining firm forecasting, but rather prefer to distribute the cost to individual plant owners, who are in any case motivated to submit good forecasts, owing to the penalty schemes. In fact, many power systems have already seen increasing interest in pairing energy storage with distributed power systems [
Both the BU- and MinT-reconciled forecasts at are used to study firm forecasting at the plant level. It should be noted that the BU forecasts at are no different from the raw 3Tier forecasts.

Fig. 6 Differences in $/kW premium of firm forecasting based on 3Tier and MinT-reconciled forecasts at individual plant level.
To compare the results of firm forecasting at the individual plant level with that at the substation level, the firm premium, overbuilding ratio, battery capacity, and $/kW premium at the individual plants are aggregated to their respective substations, of which the results are listed in
Index | Capacity (MW) | Firm premium | Overbuilding ratio | Battery capacity (MWh) | $/kW premium | ||||
---|---|---|---|---|---|---|---|---|---|
BU | MinT | BU | MinT | BU | MinT | BU | MinT | ||
1 | 171 | 3.13 | 2.53 | 1.83 | 1.59 | 1362 | 885 | 201 | 135 |
2 | 136 | 4.57 | 3.36 | 2.78 | 2.21 | 1783 | 841 | 366 | 205 |
3 | 225 | 2.34 | 2.41 | 1.57 | 1.65 | 761 | 867 | 105 | 119 |
4 | 759 | 2.71 | 2.74 | 1.66 | 1.75 | 3823 | 3809 | 139 | 146 |
5 | 965 | 1.99 | 2.07 | 1.44 | 1.28 | 3135 | 3922 | 91 | 91 |
6 | 813 | 2.62 | 2.69 | 1.62 | 1.67 | 4491 | 4290 | 143 | 143 |
7 | 7 | 2.91 | 2.35 | 1.65 | 1.15 | 57 | 45 | 189 | 120 |
8 | 630 | 2.50 | 2.11 | 1.56 | 1.39 | 3318 | 2373 | 134 | 96 |
9 | 1866 | 2.53 | 2.29 | 1.89 | 1.64 | 7229 | 6412 | 139 | 111 |
10 | 121 | 2.72 | 2.31 | 1.47 | 1.22 | 855 | 698 | 155 | 114 |
11 | 1429 | 2.10 | 2.13 | 1.38 | 1.26 | 6099 | 6442 | 104 | 98 |
12 | 650 | 2.89 | 2.48 | 1.72 | 1.42 | 4481 | 3650 | 173 | 129 |
13 | 966 | 2.14 | 2.17 | 1.40 | 1.31 | 4166 | 4315 | 106 | 101 |
14 | 68 | 4.10 | 2.80 | 1.31 | 1.10 | 1030 | 595 | 272 | 152 |
15 | 2564 | 3.00 | 2.84 | 2.49 | 2.18 | 9760 | 9479 | 189 | 161 |
16 | 1290 | 2.23 | 2.26 | 1.62 | 1.47 | 4885 | 5277 | 116 | 108 |
17 | 1005 | 2.37 | 2.46 | 1.38 | 1.29 | 5413 | 6053 | 122 | 124 |
18 | 652 | 1.98 | 2.12 | 1.30 | 1.26 | 2551 | 2912 | 91 | 96 |
19 | 1928 | 2.05 | 2.11 | 1.30 | 1.31 | 8154 | 7939 | 96 | 95 |
20 | 306 | 2.47 | 2.64 | 1.33 | 1.48 | 1940 | 1833 | 133 | 139 |
21 | 482 | 2.30 | 2.32 | 1.50 | 1.32 | 2281 | 2505 | 121 | 113 |
22 | 451 | 2.89 | 2.74 | 2.04 | 1.82 | 2321 | 2186 | 172 | 149 |
23 | 11 | 3.73 | 3.45 | 2.38 | 1.68 | 86 | 99 | 246 | 207 |
24 | 224 | 2.94 | 2.87 | 1.90 | 1.81 | 1794 | 1374 | 210 | 170 |
25 | 112 | 3.04 | 2.79 | 1.86 | 1.64 | 882 | 691 | 203 | 156 |
26 | 1126 | 2.05 | 2.25 | 1.40 | 1.30 | 4315 | 5588 | 99 | 108 |
27 | 332 | 4.11 | 2.58 | 3.01 | 1.71 | 3490 | 1586 | 344 | 139 |
28 | 465 | 2.31 | 2.25 | 1.47 | 1.29 | 2267 | 2301 | 121 | 106 |
29 | 378 | 3.24 | 2.78 | 1.71 | 1.63 | 3644 | 2367 | 219 | 157 |
30 | 36 | 2.58 | 2.43 | 2.14 | 1.54 | 117 | 173 | 151 | 126 |
31 | 64 | 3.74 | 2.39 | 3.04 | 1.80 | 470 | 214 | 293 | 122 |
32 | 531 | 2.55 | 2.52 | 1.61 | 1.58 | 2883 | 2733 | 142 | 134 |
33 | 169 | 2.43 | 2.65 | 1.57 | 1.41 | 772 | 1098 | 123 | 141 |
34 | 1571 | 1.95 | 2.14 | 1.39 | 1.29 | 4943 | 6769 | 85 | 96 |
Overall | 22503 | 2.59 | 2.41 | 1.67 | 1.52 | 105559 | 102319 | 135 | 119 |

Fig. 7 Comparison of $/kW premium of firm forecasting at individual plant level and at substation level. (a) BU. (b) MinT.
The main contribution of this paper is to integrate the concepts of hierarchical forecasting and firm forecasting into the existing solar power forecasting framework that is widely supported by grid integration policies. Existing solar forecasting for grid integration purposes consists of three steps: ① national weather centers issue NWP forecasts to PV system owners and solar forecast providers, ② the weather forecast users post-process the received forecasts based on local information, and ③ the post-processed weather forecasts are converted to PV power output forecasts and submitted to grid operators. These three steps depict a two-way information flow among various entities relevant to grid integration. Moving beyond the status quo, grid operators should reconcile the received forecasts with their own nodal- or substation-level forecasts, to enhance the quality of forecasts as well as to obtain a set of aggregate consistent forecasts that is conducive to decision-making as a whole. Since reconciled forecasts still contain sizable errors that limit the confidence in the subsequent power system operations, firming up the forecasts through battery storage and PV overbuilding is an attractive course of action, as such the PV plants can guarantee to deliver the same amount of energy as they forecast to deliver, making them effectively dispatchable.
Through a case study with 405 PV plants in California, USA, several findings emerge. Most important among those is that performing firm forecasting at the individual plant level is less economic than that at the nodal or substation level. This conclusion agrees with the implications of the widely known effect of geographical smoothing, which is another major firm forecast enabler besides storage, demand response, and overbuilding. Firm forecasting at the substation level suggests the need for energy storage sharing, which is an increasingly popular research topic [
One limitation of the proposed framework is that it does not consider uncertainty. There are several sources of uncertainty that could affect the eventual quantification of the firm forecast premium. For instance, the solution of the current mix-integer linear programming (20)-(29) depends on the particular set of forecast and production time series entering the optimization. However, the PV generation and forecast errors can change substantially over different years, which can render the firm forecast premium higher or lower. Another source of uncertainty originates from continuous technology development, which may gradually lower the cost of PV and batteries over time. Therefore, it is important to consider at least three directions for future research. First, the effect of inter-annual variability in future solar irradiance and thus future PV power should be quantified. This can be done by considering climate models, which issue irradiance projections over several future decades. Second, the uncertainty of forecast errors may be further analyzed by trying different forecasting methods and over different years. Third, optimization methods that consider uncertainties, such as robust optimization or stochastic programming, should be taken into consideration. However, all three future directions for research in firm forecasting require careful design of experiments, e.g., the choice of spatial downscaling used for climate models or the distributional assumptions used in stochastic programming. It is believed that firm forecasting has growing interests and presents a promising avenue toward dispatchable solar power in modern power systems.
References
D. Yang and J. Kleissl. (2024, Feb.). Solar irradiance and photovoltaic power forecasting. [Online]. Available: https://www.taylorfrancis.com/books/mono/10.1201/9781003203971 [Baidu Scholar]
A. Faustine and L. Pereira, “FPSeq2Q: fully parameterized sequence to quantile regression for net-load forecasting with uncertainty estimates,” IEEE Transactions on Smart Grid, vol. 13, no. 3, pp. 2440-2451, May 2022. [Baidu Scholar]
D. Yang, W. Wang, C. A. Gueymard et al., “A review of solar forecasting, its dependence on atmospheric sciences and implications for grid integration: towards carbon neutrality,” Renewable and Sustainable Energy Reviews, vol. 161, p. 112348, Jun. 2022. [Baidu Scholar]
J. Remund, R. Perez, M. Perez et al., “Firm photovoltaic power generation: overview and economic outlook,” Solar RRL, vol. 7, p. 2300497, Sept. 2023. [Baidu Scholar]
Z. Liu and Y. Du, “Evolution towards dispatchable PV using forecasting, storage, and curtailment: a review,” Electric Power Systems Research, vol. 223, p. 109554, Oct. 2023. [Baidu Scholar]
G. Yang, D. Yang, M. J. Perez et al., “Hydrogen production using curtailed electricity of firm photovoltaic plants: conception, modeling, and optimization,” Energy Conversion and Management, vol. 308, p. 118356, May 2024. [Baidu Scholar]
M. Agüero, J. Peralta, E. Quintana et al., “Virtual transmission solution based on battery energy storage systems to boost transmission capacity,” Journal of Modern Power Systems and Clean Energy, vol. 12, no. 2, pp. 466-474, Mar. 2024. [Baidu Scholar]
M. Yang, L. Zhang, Y. Cui et al., “Investigating the wind power smoothing effect using set pair analysis,” IEEE Transactions on Sustainable Energy, vol. 11, no. 3, pp. 1161-1172, May 2019. [Baidu Scholar]
M. Perez, R. Perez, K. R. Rábago et al., “Overbuilding & curtailment: the cost-effective enablers of firm PV generation,” Solar Energy, vol. 180, pp. 412-422, Mar. 2019. [Baidu Scholar]
R. Perez, M. Perez, J. Schlemmer et al., “From firm solar power forecasts to firm solar power generation an effective path to ultra-high renewable penetration: a New York case study,” Energies, vol. 13, no. 17, p. 4489, Aug. 2020. [Baidu Scholar]
D. Yang, W. Li, G. M. Yagli et al., “Operational solar forecasting for grid integration: standards, challenges, and outlook,” Solar Energy, vol. 224, pp. 930-937, Aug. 2021. [Baidu Scholar]
G. Yang, D. Yang, C. Lyu et al., “Implications of future price trends and interannual resource uncertainty on firm solar power delivery with photovoltaic overbuilding and battery storage,” IEEE Transactions on Sustainable Energy, vol. 14, no. 4, pp. 2036-2048, May 2023. [Baidu Scholar]
D. Yang, “The future of solar forecasting in China,” Journal of Renewable and Sustainable Energy, vol. 15, no. 5, p. 052301, Sept. 2023. [Baidu Scholar]
M. J. Mayer and D. Yang, “Calibration of deterministic NWP forecasts and its impact on verification,” International Journal of Forecasting, vol. 39, no. 2, pp. 981-991, Apr. 2023. [Baidu Scholar]
D. Yang, W. Wang, and T. Hong, “A historical weather forecast dataset from the European Centre for Medium-Range Weather Forecasts (ECMWF) for energy forecasting,” Solar Energy, vol. 232, pp. 263-274, Jan. 2022. [Baidu Scholar]
W. Wang, H. Shi, D. Fu et al., “Moving beyond the aerosol climatology of WRF-Solar: a case study over the North China Plain,” Weather and Forecasting, vol. 39, no. 5, pp. 765-780, May 2024. [Baidu Scholar]
M. Song, D. Yang, S. Lerch et al., “Non-crossing quantile regression neural network as a calibration tool for ensemble weather forecasts,” Advances in Atmospheric Sciences, vol. 41, no. 7, pp. 1417-1437, Mar. 2024. [Baidu Scholar]
D. Yang and D. van der Meer, “Post-processing in solar forecasting: ten overarching thinking tools,” Renewable and Sustainable Energy Reviews, vol. 140, p. 110735, Apr. 2021. [Baidu Scholar]
D. Yang, X. Xia, and M. J. Mayer, “A tutorial review of the solar power curve: Regressions, model chains, and their hybridization and probabilistic extensions,” Advances in Atmospheric Sciences, vol. 41, no. 6, pp. 1023-1067, Mar. 2024. [Baidu Scholar]
M. J. Mayer and D. Yang, “Optimal place to apply post-processing in the deterministic photovoltaic power forecasting workflow,” Applied Energy, vol. 371, p. 123681, Oct. 2024. [Baidu Scholar]
M. J. Mayer, D. Yang, and B. Szintai, “Comparing global and regional downscaled NWP models for irradiance and photovoltaic power forecasting: ECMWF versus AROME,” Applied Energy, vol. 352, p. 121958, Dec. 2023. [Baidu Scholar]
M. J. Mayer and D. Yang, “Pairing ensemble numerical weather prediction with ensemble physical model chain for probabilistic photovoltaic power forecasting,” Renewable and Sustainable Energy Reviews, vol. 175, p. 113171, Apr. 2023. [Baidu Scholar]
M. J. Mayer, “Impact of the tilt angle, inverter sizing factor and row spacing on the photovoltaic power forecast accuracy,” Applied Energy, vol. 323, p. 119598, Oct. 2022. [Baidu Scholar]
M. J. Mayer and D. Yang, “Probabilistic photovoltaic power forecasting using a calibrated ensemble of model chains,” Renewable and Sustainable Energy Reviews, vol. 168, p. 112821, Oct. 2022. [Baidu Scholar]
D. Yang and J. Kleissl, “Summarizing ensemble NWP forecasts for grid operators: Consistency, elicitability, and economic value,” International Journal of Forecasting, vol. 39, no. 4, pp. 1640-1654, Oct. 2023. [Baidu Scholar]
D. Yang, H. Quan, V. R. Disfani et al., “Reconciling solar forecasts: Geographical hierarchy,” Solar Energy, vol. 146, pp. 276-286, Apr. 2017. [Baidu Scholar]
R. J. Hyndman, A. J. Lee, and E. Wang, “Fast computation of reconciled forecasts for hierarchical and grouped time series,” Computational Statistics & Data Analysis, vol. 97, pp. 16-32, May 2016. [Baidu Scholar]
S. L. Wickramasuriya, G. Athanasopoulos, and R. J. Hyndman, “Optimal forecast reconciliation for hierarchical and grouped time series through trace minimization,” Journal of the American Statistical Association, vol. 114, no. 526, pp. 804-819, Jun. 2019. [Baidu Scholar]
G. Athanasopoulos, R. A. Ahmed, and R. J. Hyndman, “Hierarchical forecasts for Australian domestic tourism,” International Journal of Forecasting, vol. 25, no. 1, pp. 146-166, Jan. 2009. [Baidu Scholar]
R. J. Hyndman, R. A. Ahmed, G. Athanasopoulos et al., “Optimal combination forecasts for hierarchical time series,” Computational Statistics & Data Analysis, vol. 55, no. 9, pp. 2579-2589, Sept. 2011. [Baidu Scholar]
J. Schäfer and K. Strimmer, “A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics,” Statistical Applications in Genetics and Molecular Biology, vol. 4, no. 1, pp. 1-32, Nov. 2005. [Baidu Scholar]
W. Wang, Y. Guo, D. Yang et al., “Economics of physics-based solar forecasting in power system day-ahead scheduling,” Renewable and Sustainable Energy Reviews, vol. 199, p. 114448, Jul. 2024. [Baidu Scholar]
M. Placek. (2022, Jun.). Lithium-ion battery pack costs worldwide between 2011 and 2030. [Online]. Available: https://www.statista.com/statistics/883118/global-lithium-ion-battery-pack-costs/ [Baidu Scholar]
M. Jaganmohan. (2022, Jun.). Average installed cost for solar photovoltaics worldwide from 2010 to 2020. [Online]. Available: https://www.statista.com/statistics/809796/global-solar-power-installation-cost-per-kilowatt/ [Baidu Scholar]
W. Wang, D. Yang, N. Huang et al., “Irradiance-to-power conversion based on physical model chain: an application on the optimal configuration of multi-energy microgrid in cold climate,” Renewable and Sustainable Energy Reviews, vol. 161, p. 112356, Jun. 2022. [Baidu Scholar]
W. J. Cole, A. Frazier, P. Donohoo-Vallett et al., “2018 standard scenarios report: a U.S. electricity sector outlook,” Tech. Rep., National Renewable Energy Laboratory, Golden, USA, Nov. 2018. [Baidu Scholar]
M. Hummon, E. Ibanez, G. Brinkman et al., “Sub-hour solar data for power system modeling from static spatial variability analysis,” in Proceedings of 2nd International Workshop on Integration of Solar Power in Power Systems, Lisbon, Portugal, Dec. 2012, pp. 1-5. [Baidu Scholar]
3TIER, “Development of regional wind resource and wind plant output datasets,” Tech. Rep. NREL/SR-550-47676, 3TIER, Seattle, USA, Jul. 2010. [Baidu Scholar]
R. Perez, M. Perez, M. Pierro et al., “Operationally perfect solar power forecasts: a scalable strategy to lowest-cost firm solar power generation,” in Proceedings of 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC), Chicago, USA, Jun. 2019, pp. 1-6. [Baidu Scholar]
B. Couraud, V. Robu, D. Flynn et al., “Real-time control of distributed batteries with blockchain-enabled market export commitments,” IEEE Transactions on Sustainable Energy, vol. 13, no. 1, pp. 579-591, Jan. 2022. [Baidu Scholar]
J. Li, Z. Fang, Q. Wang et al., “Optimal operation with dynamic partitioning strategy for centralized shared energy storage station with integration of large-scale renewable energy,” Journal of Modern Power Systems and Clean Energy, vol. 12, no. 2, pp. 359-370, Mar. 2024. [Baidu Scholar]