A Bayesian Game Approach for Demand Response Management Considering Incomplete Information

Xiaofeng Liu; Difei Tang; Zhicheng Dai

网刊加载中。。。

使用Chrome浏览器效果最佳，继续浏览，你可能不会看到最佳的展示效果，

确定继续浏览么?

复制成功，请在其他浏览器进行阅读

A Bayesian Game Approach for Demand Response Management Considering Incomplete Information PDF

- ORCID：
Xiaofeng Liu
✉
- ORCID：
Difei Tang
✉
- ORCID：
Zhicheng Dai
✉

School of Electrical and Automation Engineering, Nanjing Normal University, Nanjing, China； State Grid Xuzhou Power Supply Company, Xuzhou, China

Updated：2022-03-28

DOI：10.35833/MPCE.2020.000288

OUTLINE

Abstract

Residential flexible resource is attracting much attention in demand response (DR) for peak load shifting. This paper proposes a scenario for multi-stage DR project to schedule energy consumption of residential communities considering the incomplete information. Communities in the scenario can decide whether to participate in DR in each stage, but the decision is the private information that is unknown to other communities. To optimize the energy consumption, a Bayesian game approach is formulated, in which the probability characteristic of the decision-making of residential communities is described with Markov chain considering human behavior of bounded rationality. Simulation results show that the proposed approach can benefit all residential communities and power grid, but the optimization effect is slightly inferior to that in complete information game approach.

Keywords

Demand response (DR); Bayesian game; energy consumption scheduling; Markov chain

I. Introduction

THE development and implementation of new technologies and strategies for solving the energy problems are critical to meeting the increasing energy demand in all walks of life. The main concern in this field is how to alleviate the contradiction between the energy supply and demand on the premise of environmental friendliness. Many advancements have been achieved in generation side, where distributed generation is an effective way to solve the contradiction between the energy supply and demand from “source side” [

1], [2]. Except for the way from “source side”, the energy management from “load side” turns out to be an effective way to solve the contradiction with the help of advanced measurement system and communication technology in smart grid. In the background, demand response (DR) has been widely used for end consumers, especially for residential users [3], [4].

Currently, there exists abundant research on residential DR problems. For example, [

5] presents a mathematical model for the optimal energy management of a residential building and proposes a centralized energy management system framework for off-grid operation, to reduce the energy cost of household. Reference [6] proposes an energy management system for a home, which includes the optimal scheduling of electric vehicle charging and household appliances to reduce the energy cost. References [5] and [6] mainly concentrate on the optimization of single decision maker. However, in most of DR scenarios, the optimization involves multiple decision makers. The traditional centralized optimization approach is difficult to solve such decision problem, and the need for a decentralized optimization approach has been a common consensus [7], [8].

In view of this, game theory, which is excellent in solving multi-player decision problem, has been widely employed in the field of DR [

9]-[11]. Based on the degree of game information publicity, game-theoretic approaches can be classified into complete information game approaches and incomplete information game approaches. In the complete information game approach, the information of all players is shared, while in the incomplete information game approach, partial information of players is not public [12], [13]. Presently, many works have been done on DR optimization with complete information game approach. A dynamic non-cooperative repeated game approach in [14] is utilized as the decentralized approach to optimize the energy consumption and energy trading amounts for the next day. A Stackelberg model is formulated between DR aggregator and electricity generators, in which the DR aggregator plays as the leader to optimize the bidding strategy, and electricity generators play as the followers to maximize their own profits [15]. However, it lacks of systematic research on DR with incomplete information game approach. Reference [16] proposes a scenario, where the real-time demand and price are considered in the incomplete information game approach due to the packet loss in the communication. Reference [17] proposes a non-cooperative game approach of incomplete information that captures the uncertainties on both the operator and user sides. However, the probability characteristic is not fully considered in [16] and [17], which is a significant part in the incomplete information game approach.

According to the deficiency of current research, this paper proposes a repeated game approach considering incomplete information to schedule the energy consumption of users, which considers the construction of probability characteristic with Markov chain. Considering that a DR project may be implemented for many years, some users may participate in the DR while some may exit the project during the period. In order to facilitate the management and improve the efficiency, in the proposed scenario, we assume that the DR project is divided into different stages. The duration time of each stage can be one week or one month. At the end of each stage, residential community can re-decide whether to participate in the DR in the next stage. Additionally, this paper adopts a kind of agent mechanism, in which residential community is deployed as an agent of native users to participate in the DR decision. In the proposed scenario, the decision of one residential community in each stage is private information that is unknown to other residential communities, and one residential community only knows the decisions of other residential communities in the previous stage. Communities with different decisions will have different energy consumption arrangements that will cause the change of real-time demand and price.

Since each residential community does not know the decisions of its opponents, it is difficult to predict the energy demand and price. Consequently, it is difficult for each residential community to schedule its own energy consumption with complete information game approach. To solve the issue, each residential community needs to speculate the decisions of its opponents according to the probability characteristic of decisions in the previous stage. And then, each residential community can schedule its energy consumption by considering the optimal scheduling strategies of other residential communities with different decisions. In summary, the contributions of this paper are as follows.

1) A scenario is proposed for DR project to schedule energy consumption of residential communities considering the incomplete information, in which the dynamic decision-making process of residential community on whether to participate in DR is considered in multi-stage DR project.

2) The probability characteristic of the decision-making of residential communities is described with Markov chain, in which the absolute rationality assumption for residential community is removed and the human behavior of bounded rationality, i.e., imitation and randomness, is considered.

3) A Bayesian game approach is proposed for the proposed scenario to reduce the daily cost with the formulated probability characteristic of the decision-making of residential communities, and the existence of Bayesian Nash equilibrium is proven mathematically.

The rest of this paper is organized as follows. The system model is introduced in Section II. In Section III, the Bayesian game approach is formulated and the existence of Bayesian Nash equilibrium is proven. Then, the probability characteristics for decision-making and Bayesian Nash equilibrium are analyzed in Section IV. The simulation results are presented in Section V. Finally, the conclusions are drawn in Section VI.

II. System Model

The proposed scenario for residential communities participating in DR is shown in Fig. 1. There are N residential communities, whose energy demand is provided by public power grid. The DR center, which is a service department of power grid, is employed as an information exchange system, collecting desired demand information of each residential community to public power grid and broadcasting DR information such as energy price policy to each residential community. The community center is mainly responsible for the information interaction with the DR center and the energy management of residential loads. Residential loads are divided into non-shiftable load, e.g., lamp, television, and refrigerator, and shiftable load, e.g., electric vehicle, washing machine, and dishwasher, in which only shiftable load can be scheduled by the community center.

Fig. 1 Scenario for residential community participating in DR.

When DR project is implemented, the DR center will broadcast DR information to all communities. Based on the broadcasted DR information, each residential community who is willing to participate in DR independently executes the scheduling algorithm to obtain the optimal energy consumption. Those communities who are unwilling to participate in DR will consume energy in their own ways. After all communities finish the scheduling, they send the desired demand information to the DR center. Then, the DR center sends the desired demand information of all communities to public power grid. Since residential communities in the scenario cannot communicate with each other, there are no privacy issue and heavy network traffic. In addition, whether a residential community is willing to participate in DR in the next stage is generally related with the decision in current stage. Hence, in the scenario, we assume that the decision-making of each residential community on whether to participate in DR in each stage has Markov property.

The two-state Markov chain for residential community is shown in Fig. 2, where $α$ is the probability that a residential community is infected to participate in DR by each neighboring residential community who has participated in DR; $β$ is the probability that a residential community decides to exit DR; N₁ is the number of neighbors of a residential community who participates in DR in the stage s; and ${(1 - α)}^{N_{1}}$ is the probability that a residential community is not infected by all neighbors. Probabilities $α$ and $β$ are the parameters to reflect the imitation behavior and random behavior of the residential community, respectively. The higher the value of $α$ or $β$ is, the more serious the imitation behavior or random behavior is. In fact, human behavior of bounded rationality is very complex and parameters $α$ and $β$ cannot reveal the bounded rational behavior from the level of the mechanism. However, whatever the mechanism is, the bounded rational behavior will finally be presented via $α$ and $β$ . Therefore, it is feasible to describe the probability characteristic of the decision-making of communities based on the two parameters.

Fig. 2 Two-state Markov chain for residential community.

Suppose that the set of N residential communities is denoted by $𝒩 = \{1,2, \dots, N\}$ and a day is divided into H time slots with $ℋ = \{1,2, \dots, H\}$ . In addition, all shiftable household appliances of residential community n are denoted by set $𝒜_{n}$ .

A. Energy Consumption Model

Assume that for any residential community $n \in 𝒩$ in time slot $h \in ℋ$ , its non-shiftable load consumes energy $l_{n, b, h}$ and its shiftable load consumes energy $l_{n, h}$ . The shiftable load is relatively insensitive to energy consumption time, which can be shifted in a certain time interval. Suppose that any shiftable household appliance $a \in 𝒜_{n}$ consumes energy $x_{n, a, h}$ in time slot h, and it has to satisfy (1).

\sum_{h = ξ_{n, a}}^{λ_{n, a}} x_{n, a, h}^{} = E_{n, a}

(1)

where $[ξ_{n, a}, λ_{n, a}]$ is the shiftable time interval of appliance a; and E_n_,_a is the daily energy demand of appliance a. Therefore, the whole energy consumption of residential community n is expressed as:

\{\begin{array}{l} l_{n, h}^{} = \sum_{a \in 𝒜_{n}} x_{n, a, h}^{} \\ L_{n, h}^{} = l_{n, b, h}^{} + l_{n, h}^{} \end{array}

(2)

where L_n_,_h is the whole energy consumption of residential community n in time slot h. Accordingly, the set of the feasible energy consumption scheduling corresponding to residential community n can be expressed as:

𝒳_{n, a} = \{x_{n, a} |\sum_{h = ξ_{n, a}}^{λ_{n, a}} x_{n, a, h}^{} = E_{n, a}; x_{n, a, h}^{} = 0, \forall h \in ℋ \ [ξ_{n, a}, λ_{n, a}]\}

(3)

where $x_{n, a} = [x_{n, a, 1}, x_{n, a, 2}, \dots, x_{n, a, H}]$ is the scheduling vector of appliance a.

B. Energy Price Model

It is clear that the energy price mechanism plays an important role in attracting more residential communities to participate in DR. In recent years, various price mechanisms have been studied, including time-of-use price [

18], real-time price [19], and critical-peak price. In this paper, the real-time price is adopted as the financial settlement between the public power grid and residential communities, i.e.,

p_{h} (L_{h}) = k_{1, h}^{} L_{h} + k_{2, h}^{}

(4)

L_{h} = \sum_{n = 1}^{N} L_{n, h}^{}

(5)

where $p_{h} (\cdot)$ is the function of real-time price; $k_{1, h} > 0$ and $k_{2, h} > 0$ are the coefficients of energy price with higher values during peak hours; and L_h is the whole energy consumption of all communities.

Based on the energy price model (4), the cost of each residential community can be calculated. Accordingly, the total daily cost of residential community n can be calculated as:

C_{n} = \sum_{h = 1}^{H} p_{h} (L_{h}) L_{n, h}^{}

(6)

The objective of the residential community participating in DR is to minimize the daily cost, i.e.,

\underset{x_{n, a} \in X_{n, a}, \forall a \in 𝒜_{n}}{m i n} C_{n} (x_{n, a})

(7)

Residential communities can obtain the optimal result by solving the optimization problem (7).

III. Bayesian Game Approach Among Residential Communities

In this section, the complete information game approach will be firstly formulated among residential communities, in which the game information of each residential community is known to all communities. Then, the Bayesian game approach for the energy consumption is formulated considering the incomplete information.

A. Formulation of Complete Information Game Approach

In the complete information game approach, the DR information of each residential community such as the initial energy consumption in each time slot and the decision on whether to participate in DR is well known to all communities. Consequently, each residential community will try to minimize its daily cost by speculating the scheduling strategies of other communities. Therefore, according to (6), the complete information game approach among residential communities can be formulated as follows.

1) Players: all communities who are willing to participate in DR.

2) Strategies: each residential community n schedules its energy consumption of shiftable load x_n_,_a to minimize the daily cost.

3) Payoffs: the payoff of residential community n is defined as:

P_{n} (x_{n, a}, x_{- n, a}) = - C_{n} (x_{n, a}, x_{- n, a})

(8)

where $x_{- n, a} = [x_{1, a}, x_{2, a}, \dots, x_{n - 1, a}, x_{n + 1, a}, \dots, x_{N, a}]$ is the scheduling strategies of other communities except residential community n.

It needs to be noted that, the above formulated game approach is mainly designed for the communities participating in DR. For some communities who do not participate in DR, they just need to consume energy in their initial state and the scheduling strategy x_n_,_a is assigned with initial energy consumption. When these communities schedule energy consumption to maximize their own payoff based on the strategies of opponents until all strategies are unchanged, such state is called Nash equilibrium. Assume that $(x_{n, a}^{*}, x_{- n, a}^{*})$ is the corresponding Nash equilibrium of the formulated complete information game approach, we can obtain:

P_{n} (x_{n, a}^{*}, x_{- n, a}^{*}) \geq P_{n} (x_{n, a}^{}, x_{- n, a}^{*})

(9)

Once the Nash equilibrium is reached, no residential community will break such equilibrium state. Otherwise its payoff will be reduced.

B. Formulation of Bayesian Game Approach

In the complete information game approach, each residential community has the full information of other communities and can speculate the scheduling strategies of other communities via solving the optimization problem (8). However, a lot of information in reality is not public information. For example, in the proposed scenario, each residential community only knows the decision of opponents on whether to participate in DR in the historical stage, and it is not easy to know the decision of opponents in the future stage. That is to say, when the residential community participates in energy management, it is not easy to know if other communities participate in DR in the next stage. Consequently, one residential community cannot speculate the scheduling strategies of other communities because the payoff function is unknown completely. Hence, the modeling process of the complete information game approach is not suitable for the incomplete information game approach. In this paper, Bayesian game approach is employed to describe the competition behavior among residential communities considering the incomplete information. Different from the complete information game approach, the basic elements of Bayesian game approach have to introduce the types of players and the probability distribution of the types except for players, strategies, and payoffs.

Assume that residential communities are divided into two types according to the decision on whether to participate in DR. That is, the type space T_n of residential community n has $| T_{n} | = 2$ elements and the actual type of residential community n is t_n. Herein, let $t_{n} = 1$ represent that the residential community is willing to participate in DR, otherwise $t_{n} = 2$ . Accordingly, $T = T_{1} \times T_{2} \times \dots \times T_{N}$ represents the type space combination for all communities and the actual type combination of all communities is $t = [t_{1}, t_{2}, \dots, t_{N}]$ . Since each residential community does not know the types of its opponents, it needs to speculate the types of opponents according to the probability distribution of the types. On the basis of Bayesian formula, we can obtain:

P r (t_{- n} | t_{n}) = \frac{P r (t_{n}, t_{- n})}{P r (t_{n})} = \frac{P r (t_{n}, t_{- n})}{\sum_{t_{- n} \in T_{- n}} P r (t_{n}, t_{- n})}

(10)

where $T_{- n} = T_{1} \times \dots \times T_{n - 1} \times T_{n + 1} \times \dots \times T_{N}$ and $t_{- n}$ are the type combination for other $N - 1$ communities except residential community n and the actual type combination of these communities, respectively; $P r (t_{- n} | t_{n})$ is the conditional probability of $t_{- n}$ under the condition that the type of residential community n is t_n; $P r (t_{n})$ is the probability that the type of residential community n is t_n; and $P r (t_{n}, t_{- n}) = P r (t)$ is the joint probability distribution for type combination t.

It is clear that, the types of all communities can be deduced from the prospect of probability with the Bayesian condition probability (10). In other words, the incomplete information game can be translated into various complete information games by dividing different type combinations of all communities and each complete information game appears with a certain probability. Therefore, the payoff of the Bayesian game is actually the expected value of all payoffs of these complete information games. When the type of residential community n is t_n, it will speculate the type combination of opponents with Bayesian formula $P r (t_{- n} | t_{n})$ and then formulate the payoff function of Bayesian game. Accordingly, the payoff of Bayesian game of residential community n with type t_n can be expressed as:

E P_{n} (t_{n}) = \sum_{t_{- n} \in T_{- n}} P_{n} (t_{n}, x_{n, a} (t_{n}), x_{- n, a} (t_{- n})) P r (t_{- n} | t_{n})

(11)

where $x_{n, a} (t_{n})$ is the scheduling strategy of residential community n with type t_n; and $x_{- n, a} (t_{- n})$ is the scheduling strategies of other communities with type combination $t_{- n}$ . Equation (11) is the objective function of residential community n with type $t_{n}$ , and the optimal strategy $x_{n, a} (t_{n})$ can be obtained by solving the maximization problem (11). When the scheduling strategies of all communities are unchanged, such equilibrium is called Bayesian Nash equilibrium, which is defined as:

E P_{n} (x_{n, a}^{*} (t_{n}), x_{- n, a}^{*} (t_{- n})) \geq E P_{n} (x_{n, a}^{} (t_{n}), x_{- n, a}^{*} (t_{- n}))

(12)

where $[x_{n, a}^{*} (t_{n}), x_{- n, a}^{*} (t_{- n})]$ is the Bayesian Nash equilibrium corresponding to type combination $t = [t_{n}, t_{- n}]$ . It needs to point out that, this paper focuses on the multi-stage DR, hence, the Bayesian game approach in this section is formulated for any DR stage. In order to have a better expression, the identifier “s” for DR stage is omitted in the above analysis, but it will be added in the analysis of joint probability distribution in next section.

IV. Solution for Bayesian Game Approach

Based on the formulated Bayesian game approach, one can know that the determination of joint probability distribution is significant for the equilibrium solution. Therefore, this section mainly focuses on the analysis of joint probability distribution and Bayesian Nash equilibrium.

A. Markov Model for Joint Probability Distribution

In the proposed scenario, the implement process of the DR project is divided into different stages. At the end of each stage, residential community can freely decide whether to participate in DR in the next stage. Since a residential community whether to participate in DR in next stage is only related with the current stage, the decision-making process has the Markov property [

20], i.e.,

P r (t_{n} (s + 1) | t_{n} (1), t_{n} (2), \dots, t_{n} (s)) = P r (t_{n} (s + 1) | t_{n} (s))

(13)

where $t_{n} (1), t_{n} (2), \dots, t_{n} (s)$ are the actual type of residential community n from stage 1 to stage s; and $P r (t_{n} (s + 1) | t_{n} (s))$ is the probability of residential community n being type $t_{n} (s + 1)$ under the condition of being type $t_{n} (s)$ . Before Bayesian game approach is applied, the state of each residential community has to be determined separately. In fact, the determination of $P r (t_{n} (s + 1) | t_{n} (s))$ can be obtained from Fig. 2. According to Fig. 2, there are four cases in total.

1) Case 1: residential community n participates in DR in stage s, i.e., $t_{n} (s) = 1$ , and still participates in DR in stage $s + 1$ , i.e., $t_{n} (s + 1) = 1$ . The corresponding conditional probability is expressed as:

P r (t_{n} (s + 1) | t_{n} (s)) = 1 - β

(14)

2) Case 2: residential community n participates in DR in stage s, i.e., $t_{n} (s) = 1$ , and exits DR in stage s+1, i.e., $t_{n} (s + 1) = 2$ . The corresponding conditional probability is expressed as:

P r (t_{n} (s + 1) | t_{n} (s)) = β

(15)

3) Case 3: residential community n does not participate in DR in stage s, i.e., $t_{n} (s) = 2$ and participates in DR in stage $s + 1$ , i.e., $t_{n} (s + 1) = 1$ . The corresponding conditional probability is expressed as:

P r (t_{n} (s + 1) | t_{n} (s)) = 1 - {(1 - α)}^{N_{1}}

(16)

4) Case 4: residential community n does not participate in DR in stage s, i.e., $t_{n} (s) = 2$ and does not participate in DR in stage $s + 1$ as well, i.e., $t_{n} (s + 1) = 2$ . The corresponding conditional probability is expressed as:

P r (t_{n} (s + 1) | t_{n} (s)) = {(1 - α)}^{N_{1}}

(17)

Therefore, the transition probability of Markov chain for residential community n can be expressed as:

P r (t_{n} (s + 1) | t_{n} (s)) = \{\begin{array}{l} 1 - β (t_{n} (s), (t_{n} (s + 1)) = (1,1) \\ β (t_{n} (s), (t_{n} (s + 1)) = (1,2) \\ 1 - {(1 - α)}^{N_{1}} (t_{n} (s), (t_{n} (s + 1)) = (2,1) \\ {(1 - α)}^{N_{1}} (t_{n} (s), (t_{n} (s + 1)) = (2,2) \end{array}

(18)

According to the characteristic and the transition probability of Markov chain, we can obtain the probability of residential community n with type 1 in stage $s + 1$ as:

P r_{n} (t_{n} (s + 1) = 1) = {[\begin{matrix} P r_{n} (t_{n} (s) = 1) \\ P r_{n} (t_{n} (s) = 2) \end{matrix}]}^{T} [\begin{array}{l} P r (t_{n} (s + 1) | t_{n} (s) = 1) \\ P r (t_{n} (s + 1) | t_{n} (s) = 2) \end{array}]

(19)

Equation (19) can be rewritten as:

P r_{n} (t_{n} (s + 1) = 1) = (1 - β) P r_{n} (t_{n} (s) = 1) + [1 - {(1 - α)}^{N_{1}}] P r_{n} (t_{n} (s) = 2)

(20)

Assume that $𝒥$ is the set of all neighbors of residential community n and residential community j is any one in the set $𝒥$ . Clearly, residential community j will participate in DR with the probability $P r_{j} (t_{j} (s) = 1)$ and not participate in DR with the probability $P r_{j} (t_{j} (s) = 2)$ . When residential community j participates in DR, the probability that residential community n is not infected is equal to $1 - α$ ; otherwise, the corresponding probability is equal to 1. That is, the probability that residential community n is not infected by residential community j can be expressed as:

(1 - α) P r_{j} (t_{j} (s) = 1) + P r_{j} (t_{j} (s) = 2)

(21)

Accordingly, the probability that residential community n is not infected by all neighbors can be rewritten as:

\begin{array}{l} {(1 - α)}^{N_{1}} = \prod_{j \in 𝒥} [(1 - α) P r_{j} (t_{j} (s) = 1) + P r_{j} (t_{j} (s) = 2)] = \\ \prod_{j \in 𝒥} (1 - α P r_{j} (t_{j} (s) = 1)) \approx 1 - α \sum_{j \in 𝒥} P r_{j} (t_{j} (s) = 1) \end{array}

(22)

Therefore, the probability of residential community n with type 1 and type 2 can be expressed as:

\begin{array}{l} P r_{n} (t_{n} (s + 1) = 1) = (1 - β) P r_{n} (t_{n} (s) = 1) + \\ [1 - (1 - α \sum_{j \in 𝒥} P r_{j} (t_{j} (s) = 1))] P r_{n} (t_{n} (s) = 2) \end{array}

(23)

\begin{array}{l} P r_{n} (t_{n} (s + 1) = 2) = β P r_{n} (t_{n} (s) = 1) + \\ [1 - α \sum_{j \in 𝒥} P r_{j} (t_{j} (s) = 1)] P r_{n} (t_{n} (s) = 2) \end{array}

(24)

Considering residential communities are in a well-connected information network, the probability characteristic of the decisions of communities on whether to participate in DR in previous stage is the public information in the network. Therefore, the set $𝒥$ can be considered as set $𝒩$ \n, then the probability of residential community n with type 1 and type 2 is equal to (25) and (26), respectively.

\begin{array}{l} P r_{n} (t_{n} (s + 1) = 1) = (1 - β) P r_{n} (t_{n} (s) = 1) + \\ α \sum_{j \in 𝒩 \ n} P r_{j} (t_{j} (s) = 1) P r_{n} (t_{n} (s) = 2) \end{array}

(25)

P r_{n} (t_{n} (s + 1) = 2) = 1 - P r_{n} (t_{n} (s + 1) = 1)

(26)

Based on (25) and (26), the probability of each residential community with type 1 or 2 in stage $s + 1$ can be easily obtained. Consequently, the joint probability distribution $P r (t)$ will be determined for all type combinations. Furthermore, Bayesian conditional probability will be obtained with (10).

B. Bayesian Nash Equilibrium

According to the above analysis, it is obvious that Bayesian Nash equilibrium is closely correlated with the type of players. Hence, the residential community will have different Bayesian Nash equilibriums for different types. In this subsection, the existence and uniqueness of the Bayesian Nash equilibrium will be proven for any actual type combination t.

Proposition 1: in the formulated complete information game approach, the Bayesian Nash equilibrium is unique.

Proof: obviously, the payoff function $P_{n} (x_{n, a}, x_{- n, a})$ is continuously differentiable in $x_{n, a}$ for the fixed $x_{- n, a}$ . Hence, the Hessian matrix of function $P_{n} (x_{n, a}, x_{- n, a})$ can be obtained as:

\begin{array}{l} \nabla_{x_{n, a}}^{2} P_{n} (x_{n, a}, x_{- n, a}) = - d i a g \{{\ddot{p}}_{h} \sum_{a \in 𝒜_{n}} x_{n, a, h}^{} + 2 {\dot{p}}_{h}\} = d i a g {- 2 k_{1, h}^{}} \\ h = 1,2, \dots, H \end{array}

(27)

Due to the coefficient of energy price $k_{1, h} > 0$ , $\nabla_{x_{n, a}}^{2} P_{n} (x_{n, a}, x_{- n, a})$ is a diagonal matrix with all negative elements. Therefore, the function $P_{n} (x_{n, a}, x_{- n, a})$ is concave in $x_{n, a}$ . Consequently, the existence of Nash equilibrium can be proven according to Theorem 1 in [

21], and the uniqueness of Nash equilibrium can be proven according to Theorem 3 in [21]. According to Proposition 1, we can easily obtain the following proposition.

Proposition 2: in the formulated Bayesian game approach, Bayesian Nash equilibrium is unique for any actual type combination t.

Proof: similarly, we just need to prove the concavity of payoff function $E P_{n} (t_{n})$ . Accordingly, the corresponding Hessian matrix is expressed as:

\nabla_{x_{n, a} (t_{n})}^{2} E P_{n} (t_{n}) = \sum_{t_{- n} \in T_{- n}} \nabla_{x_{n, a}}^{2} P_{n} (x_{n, a} (t_{n}), x_{- n, a} (t_{- n})) P r (t_{- n} | t_{n})

(28)

Since $\sum_{t_{- n} \in T_{- n}} P r (t_{- n} | t_{n}) = 1$ , (28) can also be rewritten as:

\nabla_{x_{n, a} (t_{n})}^{2} E P_{n} (t_{n}) = d i a g {- 2 k_{1, h}^{}} h = 1,2, \dots, H

(29)

Accordingly, the Bayesian Nash equilibrium is unique for any actual type combination t.

In order to search the optimal solution of the Bayesian game approach, a distributed algorithm executed by community center is proposed, which is shown in Algorithm 1.

Algorithm 1
Input: energy price parameters, initial energy consumption, initial probability distribution of decisions of residential communities on whether to participate in DR
Output: optimal arrangement of shiftable load
1. Calculate the probability of any residential community with type 1 or 2 in stage $s + 1$ according to the type of residential communities in stage s
2. Calculate the joint probability distribution of \|T\| type combinations in stage $s + 1$
3. Repeat
4. $n = 1$
5. for $n ⩽ N$ do
6. if $t_{n} (s + 1) = 1$
7. Update the strategy of residential community n $x_{n, a} (t_{n})$ by solving the maximization problem (11) 8. else
9. Update the strategy of residential community n $x_{n, a} (t_{n})$ with initial consumption
10. end
11. $n = n + 1$
12. end
13. Until no residential community changes its strategy
14. Return strategy $x_{n, a} (t_{n})$ of residential community n with type t_n

In stage 7, given $x_{- n, a} (t_{- n})$ , the problem (11) has only local variable $x_{n, a} (t_{n})$ and can be solved with the mature mathematic programming algorithm. In this paper, CPLEX optimization solver is adopted, which has a high convergence and efficiency. Additionally, for the Bayesian Nash equilibrium, each residential community has to constantly readjust its strategy until the equilibrium is reached. The dynamic decision-making process of the Bayesian game can be summarized as follows.

1) Community n calculates the optimal scheduling strategies of other $N - 1$ residential communities one by one according to its own strategy.

2) Community n updates the original strategy according to the new scheduling strategies of other $N - 1$ residential communities.

3) Repeat 1) and 2) until the equilibrium is achieved.

Such dynamic process is realized with stages 3-12. Since the formulated game model has Nash equilibrium, Algorithm 1 will converge to the equilibrium by executing stages 3-12 [

22], [23]. Here, it needs to note that, all residential communities who are willing to participate in DR have the same process. In other word, each residential community will obtain an equilibrium solution by executing Algorithm 1. But, since the Bayesian Nash equilibrium of the game is unique, the obtained solution is actually the same equilibrium.

V. Simulation Results

In this section, simulation results are presented to show the effectiveness of the formulated Bayesian game approach and the performance of the designed distributed algorithm.

Assume that there are 3 residential communities and each residential community contains 800 users in the case. A day is divided into 24 time slots and each time slot is 1 hour. Initial energy consumption of each residential community before DR is given with a random energy demand value between the upper limit and the lower limit that are set in Fig. 3. Note that, the initial energy consumption considers the non-shiftable load and the shiftable load, in which the shiftable load contains washing machine, dishwasher, and electric vehicle.

Fig. 3 Energy demand range of each residential community.

Such shiftable loads can be scheduled uniformly by community center during the permitted time interval. Herein, we assume that the permitted time slots of shiftable loads are as follows: electric vehicle can be charged from 17:00 to 24:00 and 00:00 to 06:00; washing machine can operate from 17:00 to 23:00; and the dishwasher can operate from 17:00 to 23:00. In addition, considering the difference of operation characteristics of the shiftable appliances, we assume that the electric vehicle can be charged at any time during the permitted interval, while the washing machine or dishwasher only works once a day and the operation time is 1 hour [

24]. According to Fig. 3, energy price parameters are set as follows:

k_{1, h} = 1.2

k_{2, h} = 42.86

(

h = 1

, 2,

\dots

, 6);

k_{1, h} = 1.9

k_{2, h} = 71.43

(

h = 7

, 8,

\dots

, 17 and

h = 24

);

k_{1, h} = 2.5

k_{2, h} = 128.57

(

h = 18

, 19,

\dots

, 23). Furthermore, suppose that in the initial stage of conducting DR project, only a third of residential communities are willing to participate in DR. That is, the probability distribution of decisions of residential communities in stage

s = 1

is equal to

P r_{n} (t_{n} (1) = 1) = 1 / 3

and

P r_{n} (t_{n} (1) = 2) = 2 / 3

(

n = 1

, 2, 3). In the future stage, we assume that a residential community can be infected to participate in DR by each residential community with probability

α = 0.2

, and the residential community will have the probability

β = 0.1

from type 1 to type 2.

A. Optimal Strategy of Residential Community

Since each stage has an equilibrium solution, this subsection will take the result of stage $s = 2$ as an example. According to (25) and (26), the probability of any residential community n with type 1 is equal to $P r_{n} (t_{n} (2) = 1) = 0.389$ and $P r_{n} (t_{n} (2) = 2) = 0.611$ . Since 3 residential communities are considered, there are 2³ type combinations. Therefore, the joint distribution probability for $t = [t_{1}, t_{2}, t_{3}]$ is equal to:

P r (t) = 0 . 389^{M} \cdot 0 . 611^{3 - M}

(31)

where $M = 1$ , 2, 3 represents the number of residential communities with type 1 in actual type combination $t = [t_{1}, t_{2}, t_{3}]$ . Additionally, we assume that the actual type combination of 3 residential communities is $t = [1,1, 1]$ in stage $s = 2$ . Based on the above simulation parameters, the equilibrium can be obtained by executing Algorithm 1.

Tables I and II show the optimal operation strategy of shiftable appliances in each time slot in the complete information game and Bayesian game, respectively. In the tables, “DW”, “WM”, and “EV” represent the dishwasher, washing machine and electric vehicle, respectively; the identifier “√” represents that the appliance will consume energy in the corresponding time slot.

TABLE I Optimal Operation Strategy of Appliances in Complete Information Game Approach

Time slot	Residential community 1			Residential community 2			Residential community 3
Time slot	DW	WM	EV	DW	WM	EV	DW	WM	EV
1-6			√			√			√
7-17
18							√
19								√
20		√
21					√
22	√
23				√
24

TABLE II Optimal Operation Strategy of Appliances in Bayesian Game Approach

Time slot	Residential community 1			Residential community 2			Residential community 3
Time slot	DW	WM	EV	DW	WM	EV	DW	WM	EV
1-6			√			√			√
7-17
18	√						√
19
20
21		√		√
22
23					√			√
24

From the tables, it can be seen that electric vehicles in 3 residential communities are all shifted to the off-peak hours, i.e., time slots 1-6, but the operation strategies of the dishwasher and washing machine are different between the two approaches. In the complete information game approach, the operation time of dishwasher and washing machine in 3 residential communities are scheduled into 6 time slots from time slot 18 to time slot 23. That is, each time slot only has one shiftable appliance, for example, time slot 18 only has the dishwasher of residential community 3. However, in Bayesian game approach, some time slots have more than one shiftable appliance, for example, time slot 21 has the washing machine of residential community 1 and the dishwasher of residential community 2. The main reason for the difference is that, since the decision of each residential community on whether to participate in DR in stage 2 is unknown to other communities, each residential community cannot deduce the strategy of its opponents precisely. Consequently, residential community can only make a compromise strategy to maximize the expected payoff by considering all possible type combinations.

B. Benefits of Residential Community and Grid

Residential community and public power grid can effectively obtain the benefits from the proposed Bayesian game approach. However, due to the lack of game information, the obtained benefits in the Bayesian game are less than those in the complete information game.

Figure 4 is the optimal energy demand of each residential community in the Bayesian game. It can be observed that, compared with the initial energy demand, the optimal energy demand of each residential community has a good effect on load shifting.

Fig. 4 Optimal energy demand of each residential community in Bayesian game. (a) Residential community 1. (b) Residential community 2. (c) Residential community 3.

The corresponding energy demand of all communities is shown in Fig. 5. It can be observed that the maximum energy demand is 3.57 MWh in the complete information game, while the maximum energy demand is 4.05 MWh in the Bayesian game in time slot 18. The main reason is that, time slot 18 has dishwashers of two communities simultaneously in the Bayesian game. It demonstrates that the Bayesian game approach is liable to cause load aggregation.

Fig. 5 Energy demand of all communities in complete information and Bayesian game approaches. (a) Complete information game approach. (b) Bayesian game approach.

In order to quantitatively analyze the variation of energy demand, peak to average ratio (PAR) can be introduced, which is calculated as [

9]:

P A R_{g} = \frac{\underset{h \in ℋ}{m a x} L_{h}}{\frac{1}{H} \sum_{h \in ℋ} L_{h}}

(32)

P A R_{n} = \frac{\underset{h \in ℋ}{m a x} L_{n, h}}{\frac{1}{H} \sum_{h \in ℋ} L_{n, h}}

(33)

where PAR_g is the PAR in public power grid; and PAR_n is the PAR in residential community n. Basically, Fig. 6 depicts the PAR in public power grid (solid lines) and each residential community (discrete points).

Fig. 6 PAR in public power grid and each residential community.

Before DR project is implemented, no matter in public power grid or in each residential community, the PAR is very high. However, although the PAR in each residential community is high after the two game approaches have been employed, the PAR in public power grid is reduced greatly. It indicates that the two game approaches can realize the demand complementation among communities. The daily cost of each residential community is shown Fig. 7. From the optimal results of the two game approaches, it can be observed that the optimization effect of Bayesian game approach on PAR and the daily cost is slightly less than that of the complete information game approach. It needs to be pointed out that, although the disclosure of game information will contribute the optimization effect, players are still unwilling to completely disclose the private information due to the privacy protection and human selfishness.

Fig. 7 Daily cost of each residential community.

C. Evolution Analysis of Probability in DR

In the above case, for the convenience of analysis, we assume that the actual types of 3 residential communities are all type 1, but such probability is only 0.059. In addition, the above case only concerns the optimal result in a certain stage. Therefore, this subsection mainly concentrates on the evolution analysis for the probability of communities participating in DR with the execution of DR project. Specially, the different evolution results for communities with different degrees of rationality will be analyzed by regulating parameters $α$ and $β$ . Since different groups of users have various degrees of rationality in the reality, the obtained result can provide the reference in the design and implementation of DR project considering different user groups. The evolution result of the probabilities of residential communities with types 1 and 2 is shown in Fig. 8.

Fig. 8 Evolution result of probabilities of residential communities with types 1 and 2.

It depicts that the probabilities of communities with types 1 and 2 gradually converge to fixed values after 20 stages. Finally, each residential community will participate in DR with the probability of 0.75. In fact, the evolution result of the probabilities of communities is closely related with parameters N, $α$ , and $β$ . Figure 9 shows the probability of type 1 with different parameters.

Fig. 9 Probability of type 1 with different parameters. (a) $β = 0.1$ and $α = 0.2$ . (b) $N = 3$ .

It is clear that, with the growth of the number of communities, the probability of type 1 increases gradually. The main reason is that, each residential community will have a higher probability to be attracted into DR with the growth of the number of communities in DR. However, with the growth of the value of $β$ , the probability of type 1 decreases dramatically. The values of $α$ and $β$ are correlated with the DR project such as the DR price and DR experience. The growth of $β$ demonstrates that the DR project has a low attraction or experience, and the residential community is unwilling to participate in DR or wants to exit DR. Therefore, it is important to design a good DR mechanism for the better performance of DR in smart grid. According to the results in Fig. 9, the joint probability distribution of the formulated Bayesian game approach can be deduced, and then the equilibrium solution of the Bayesian game approach can be obtained.

Figure 10 shows the daily cost of 3 residential communities with different values of $β$ and $α = 0.2$ . It should be noted that the results of Fig. 10 are obtained based on the following assumptions: ① the actual number of communities participating in DR is obtained by rounding the value of the product of N and the probability with type 1; ② communities 1 and 2 participate in DR when the actual participation number is 2, and residential community 1 participates in DR when the number is 1. It can be observed that in the same case, the value of $β$ has the limited influence on the daily cost of each residential community; while in the different cases, the daily cost of each residential community has a large fluctuation. It demonstrates that the value of $β$ will affect the actual participation number of communities and the joint probability distribution, and then further affect the daily cost of each residential community. However, the daily cost of each residential community will be greatly affected with the change of the actual participation number.

Fig. 10 Daily cost of 3 residential communities with different values of $β$ and $α = 0.2$ .

Additionally, Table III shows the running time of the proposed algorithm in the cases with different numbers of communities, i.e., $N = 3$ , 4, $\dots$ , 8 and $β = 0.1$ , $α = 0.2$ . It should be noted that the actual participation number of communities is equal to N and the algorithm is conducted on the personal computer with Intel^(R) Core^(TM) i5-8500 CPU @ 3.00 GHz and RAM 8.00 GB. The table depicts that the running time of the Algorithm 1 increases gradually with the increasing number of communities. It can be foreseen that the Algorithm 1 may take longer running time for dozens or hundreds of communities on the personal computer. However, considering the finite running speed of personal computer, running time can be reduced dramatically if a computing server is used, which is acceptable in the practical application.

TABLE III Running Time of Algorithm 1

Number of residential communities	Running time (s)	Number of residential communities	Running time (s)
3	61.57	6	212.04
4	98.43	7	286.27
5	145.71	8	373.18

VI. Conclusion

In this paper, a Bayesian game approach is formulated to schedule energy consumption of shiftable load in residential communities considering the incomplete information. In the proposed scenario, each residential community can decide whether to participate in DR in any stage of DR project. The decision of each residential community is the private information, which is unknown to other communities. Therefore, each residential community needs to evaluate such information based on the probability distribution of decision. Accordingly, the Markov model for joint probability distribution is proposed to describe the decision-making process of residential community considering human behavior of bounded rationality. Simulation results demonstrate that the proposed approach can reduce the daily cost and PAR of the overall energy demand. However, compared with the complete information game approach, it shows that the optimization effect on the Bayesian game approach is weakened due to the loss of game information.

References

S. Dagoumas and N. E. Koltsaklis, “Review of models for integrating renewable energy in the generation expansion planning,” Applied Energy, vol. 242, pp. 1573-1587, May 2019. [Baidu Scholar]

V. Oree, S. Z. S. Hassen, and P. J. Fleming, “Generation expansion planning optimisation with renewable energy integration: a review,” Renewable and Sustainable Energy Reviews, vol. 69, pp. 790-803, Mar. 2017. [Baidu Scholar]

M. Yu, S. Hong, Y. Ding et al., “An incentive-based demand response (DR) model considering composited DR resources,” IEEE Transactions on Industrial Electronics, vol. 66, no. 2, pp. 1488-1498, Apr. 2018. [Baidu Scholar]

F. Elghitani and W. Zhuang, “Aggregating a large number of residential appliances for demand response applications,” IEEE Transactions on Smart Grid, vol. 9, no. 5, pp. 5092-5100, Mar. 2017. [Baidu Scholar]

I. Sharma, J. Dong, A. A. Malikopoulos et al., “A modeling framework for optimal energy management of a residential building,” Energy and Buildings, vol. 130, pp. 55-63, Oct. 2016. [Baidu Scholar]

P. Mesarić and S. Krajcar. “Home demand side management integrated with electric vehicles and renewable energy sources,” Energy and Buildings, vol. 108, pp. 1-9, Dec. 2015. [Baidu Scholar]

J. S. Vardakas, N. Zorba, and C. V. Verikoukis, “A survey on demand response programs in smart grids: pricing methods and optimization algorithms,” IEEE Communications Surveys & Tutorials, vol. 17, no. 1, pp. 152-178, Jul. 2014. [Baidu Scholar]

P. Jacquot, O. Beaude, S. Gaubert et al., “Analysis and implementation of an hourly billing mechanism for demand response management,” IEEE Transactions on Smart Grid, vol. 10, no. 4, pp. 4265-4278, Jul. 2019. [Baidu Scholar]

A. H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich et al., “Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid,” IEEE Transactions on Smart Grid, vol. 1, no. 3, pp. 320-331, Dec. 2010. [Baidu Scholar]

M. Yu and S. Hong, “Supply-demand balancing for power management in smart grid: a Stackelberg game approach,” Applied energy, vol. 164, pp. 702-710, Feb. 2016. [Baidu Scholar]

H. Chen, Y. Li, R. H. Y. Louie et al., “Autonomous demand side management based on energy consumption scheduling and instantaneous load billing: an aggregative game approach,” IEEE Transactions on Smart Grid, vol. 5, no. 4, pp. 1744-1754, Jul. 2014. [Baidu Scholar]

T. Li and M. Shahidehpour, “Strategic bidding of transmission-constrained GENCOs with incomplete information,” IEEE Transactions on Power Systems, vol. 20, no. 1, pp. 437-447, Feb. 2005. [Baidu Scholar]

X. Liu, B. Gao, C. Wu et al., “Demand-side management with household plug-in electric vehicles: a Bayesian game-theoretic approach,” IEEE Systems Journal, vol. 12, no. 3, pp. 2894-2904, Sept. 2017. [Baidu Scholar]

C. P. Mediwaththe, E. R. Stephens, D. B. Smith et al., “A dynamic game for electricity load management in neighborhood area networks,” IEEE Transactions on Smart Grid, vol. 7, no. 3, pp. 1329-1336, May 2016. [Baidu Scholar]

E. Nekouei, T. Alpcan, and D. Chattopadhyay, “Game-theoretic frameworks for demand response in electricity markets,” IEEE Transactions on Smart Grid, vol. 6, no. 2, pp. 748-758. Mar. 2015. [Baidu Scholar]

S. Misra, S. Bera, T. Ojha et al., “ENTICE: agent-based energy trading with incomplete information in the smart grid,” Journal of Network and Computer Applications, vol. 55, pp. 202-212, Sept. 2015. [Baidu Scholar]

C. Eksin, H. Delic, and A. Ribeiro, “Demand response management in smart grids with heterogeneous consumer preferences,” IEEE Transactions on Smart Grid, vol. 6, no. 6, pp. 3082-3094, Nov. 2015. [Baidu Scholar]

C. Bergaentzle, C. Clastres, and H. Khalfallah, “Demand-side management and European environmental and energy goals: an optimal complementary approach,” Energy Policy, vol. 67, pp. 858-869, Jan. 2014. [Baidu Scholar]

R. Deng, Z. Yang, J. Chen et al., “Residential energy consumption scheduling: a coupled-constraint game approach,” IEEE Transactions on Smart Grid, vol. 5, no. 3, pp. 1340-1350, May 2014. [Baidu Scholar]

M. Marzband, F. Azarinejadian, M. Savaghebi et al., “An optimal energy management system for islanded microgrids based on multiperiod artificial bee colony combined with Markov chain,” IEEE Systems Journal, vol. 11, no. 3, pp. 1712-1722, Sept. 2017. [Baidu Scholar]

B. Rosen, “Existence and uniqueness of equilibrium points for concave n-person games,” Econometrica, vol. 33, pp. 347-351, Jul. 1965. [Baidu Scholar]

Z. M. Fadlullah, D. Quan, N. Kato et al., “GTES: an optimized game-theoretic demand-side management scheme for smart grid,” IEEE Systems Journal, vol. 8, no. 2, pp. 588-597, Jun. 2013. [Baidu Scholar]

Z. Zhu, S. Lambotharan, W. Chin et al., “A game theoretic optimization framework for home demand management incorporating local energy resources,” IEEE Transactions on Industrial Informatics, vol. 11, no. 2, pp. 353-362, Apr. 2015. [Baidu Scholar]

B. Gao, W. Zhang, Y. Tang et al., “Game-theoretic energy management for residential users with dischargeable plug-in electric vehicles,” Energies, vol. 7, no. 11, pp. 7499-7518, Nov. 2014. [Baidu Scholar]

Address:No.19 Chengxin Avenue, Jiangning District, Nanjing 211106, China

E-mail: mpce@alljournals.cn

Tel:86-25-81093060

Fax:86-25-81093040

Home

Introduction

Editorial Board

For Author

Call For Papers

APC

Sponsor & Publisher