Unsupervised Learning for Non-intrusive Load Monitoring in Smart Grid Based on Spiking Deep Neural Network

Zejian Zhou; Yingmeng Xiang; Hao Xu; Yishen Wang; Di Shi

网刊加载中。。。

使用Chrome浏览器效果最佳，继续浏览，你可能不会看到最佳的展示效果，

确定继续浏览么?

复制成功，请在其他浏览器进行阅读

Unsupervised Learning for Non-intrusive Load Monitoring in Smart Grid Based on Spiking Deep Neural Network PDF

- ORCID：
Zejian Zhou
✉
- ORCID：
Yingmeng Xiang
✉
- ORCID：
Hao Xu
✉
- ORCID：
Yishen Wang
✉
- ORCID：
Di Shi
✉

Department of Electrical and Biomedical Engineering, University of Nevada, Reno, NV, USA； Global Energy Interconnection Research Institute North America (GEIRINA), San Jose, CA, USA

Updated：2022-05-11

DOI：10.35833/MPCE.2020.000569

Abstract

This paper investigates the intelligent load monitoring problem with applications to practical energy management scenarios in smart grids. As one of the critical components for paving the way to smart grids’ success, an intelligent and feasible non-intrusive load monitoring (NILM) algorithm is urgently needed. However, most recent researches on NILM have not dealt with practical problems when applied to power grid, i.e., ① limited communication for slow-change systems; ② requirement of low-cost hardware at the users’ side; and ③ inconvenience to adapt to new households. Therefore, a novel NILM algorithm based on biology-inspired spiking neural network (SNN) has been developed to overcome the existing challenges. To provide intelligence in NILM, the developed SNN features an unsupervised learning rule, i.e., spike-time dependent plasticity (STDP), which only requires the user to label one instance for each appliance while adapting to a new household. To upgrade the feasibility in NILM, the designed spiking neurons mimic the mechanism of human brain neurons that can be constructed by a resistor-capacitor (RC) circuit. In addition, a distributed computing system has been designed that divides the SNN into two parts, i.e., smart outlets and local servers. Since the information flows as sparse binary vectors among spiking neurons in the developed SNN-based NILM, the high-frequency data can be easily compressed as the spike times, and are sent to the local server with limited communication capability, whereas it is unable to handle the traditional NILM. Finally, a series of experiments are conducted using a benchmark public dataset. Meanwhile, the effectiveness of developed SNN-based NILM can be demonstrated through comparisons with other emerging NILM algorithms such as the convolutional neural networks.

Keywords

Non-intrusive load monitoring (NILM); spiking neural network (SNN); smart grid; unsupervised machine learning

I. Introduction

OVER the past decades, the emerging desire of a next-generation power grid, i.e., the smart grid [

1], has demanded a considerable amount of data to perform the more sophisticated operation of power transmission line [2] and the power demand control of individual households. As one of the essential parts in the smart grid, the power consumption profiling of household based on appliances has played a significant role. By collecting the energy consumption patterns of all homes, the utility companies are able to optimize the schedule of power transmission and balance the load demand response. In order to generate the usage patterns of customized appliances for individual households, the users have to collect and label the data of each appliance, i.e., active power, power factor, etc. However, such requirements are not feasible, which frustrates the precise profiling at the appliance level. The non-intrusive load monitoring (NILM) algorithm, which aims to classify and disaggregate appliances using numerous measurements, has attracted increasing attention because it can automatically identify the appliance type.

The basic concept of NILM is to model an appliance by its numerous electric features such as active and reactive power difference [

3], current and voltage [4], shape of waveform [5], etc. Along with the features, many previous researches have utilized different classification methods such as the hidden Markov models (HMM) [6], k-nearest neighbor (KNN) [7], etc. Moreover, due to the rapid growth of machine learning techniques, especially in image processing, recent studies of NILM are exploring the algorithms based on artificial neural networks (ANNs) for further improving the inference result. References [5] and [8] adopt long short-term memory (LSTM) to map from the aggregated power sequences to individual appliance’s load sequence. Reference [9] integrates machine learning into a wavelet design to match the active load pattern. However, due to the limit of both communication and computation resources, the data of the traditional NILM tasks are fed by a power meter, which measures the summation of power consumed by all appliances. Therefore, the previous NILM algorithms need to disaggregate the summation of power measurements before performing the classification, which greatly affects the performance.

Thanks to the emerging Internet of Things (IoT) technology, the current and voltage data of each appliance can be measured individually through a smart plug. Therefore, the burden of desegregating the summed power data is relieved, and the algorithm can focus on inferring the types of appliances only. For example, the Grid Sense system utilizes the smart outlets to measure and classify appliances [

10]. Moreover, some preliminary works have been tested with State Grid Jiangsu Electric Power Company in China [11], where the Grid Sense system has proved that the NILM algorithm can effectively help the load demand control through networked smart plugs. To further improve the classification results in the Grid Sense system, we carefully explore the possibility of utilizing deep learning method in NILM, because recent research works have shown the drastic increment of NILM classification results such as [5]. However, we have also observed that the deep learning method is questioned in many ways. Considering the hardware structure of the Grid Sense system and most techniques applied to the control of power transmission line, the main drawbacks of deep-learning based method fall on three aspects. ① The training process of neural networks requires powerful dedicated computational unit, e.g., graphics processing unit (GPU), which is not available in many low-cost IoT-embedded hardware. In NILM applications, the appliance types vary among different households. Therefore, each household is associated with unique neural networks. However, in the Grid Sense system, it is impossible to embed a powerful GPU in the smart outlet due to the large-scale deployment. As a matter of fact, the smart outlet should only contain low-cost components such as capacitors and resistors instead of high-end chips. ② One may argue that the smart outlet can collect data and then send them to the local server directly so that only the local server needs to install GPUs. However, even with the most advanced IoT technology, it is not feasible to receive real-time load data from hundreds of smart outlets at a high frequency, e.g., 30 kHz. Therefore, traditional practical NILM algorithms usually consider low-frequency data as the only option, where valuable transient information is lost. ③ The most successful deep neural network models such as the AlexNet [12] and YOLO [13] are trained in a supervised fashion, which means that users are required to collect and label enough data when the smart outlet is installed in a new household. This is not realistic for practical implementation on the Grid Sense system. The types of all appliances are enormous, and it is thus not feasible to collect enough data for every type of appliance. As one of the most investigated problems in all machine learning studies, data compression and computation cost reduction are being studied actively. In [14], a data compression technique based on auto-encoders has been used to reduce the dimension of the training set. However, the reduced dimension, e.g., 60%-95% times in [14], is still large for a high-frequency data setup. Therefore, the disadvantage of using supervised learning in NILM is obvious: ① it is difficult to obtain enough data to train the model; ② training the supervised model requires GPU, which is not cost-friendly to an emended device.

To overcome these difficulties, a novel NILM algorithm based on biology-inspired spiking neural network (SNN) feaures an unsupervised learning scheme and a distributed computing design. Recognized as the third generation of neural networks, the SNN is inspired by the human brain neurons. Human brains can encode a huge amount of information using small populations of spikes and consume significantly less energy than analog neural networks (ANNs) [

15]. Inspired by this concept, the SNN is introduced where the information flows as electric impulses (spikes) and charges the membrane potential of the connected neuron. More importantly, the spikes are sparse binary vectors. This unique setup can reduce the information flow (analog in ANN vs. binary in SNN) and is easily implemented by simple hardware. For example, in the most common SNN models, i.e., leaky-integral-fire (LIF) models, each neuron can be modeled by one capacitor and one resistor (RC circuit) only [15]. More importantly, several research works have explored the possibility of using electronic semiconductors to replace the RC circuit, which is promising to develop a delicate integrated circuit. For instance, [16] utilizes metal-oxide-semiconductor field-effect transistor (MOSFET) to construct each LIF neuron. Note that this integrated circuit for SNN is significantly different from a GPU. Instead of using chips to build a universal system such as Linux and then run the neuron model, the information in the SNN circuits is directly encoded into the physical measurements such as current and voltage, which runs faster and consumes less energy [17]. Another significant advantage of SNN is the convenience of distributed computation. In a typical ANN design, it is pointless to separate the layers into different devices, because the amount of analog data exchanged between layers increases the uncertainty of communication channels drastically. However, in SNNs, the information is coded as sparse binary vectors that are easy to compress. For example, at each time point, the output of each neuron is either 1 or 0, whereas the analog data takes 8 to 16 bits. Therefore, by using SNN, the distributed computation method becomes feasible. The high-frequency load data can be processed by the first layer installed on the smart outlet, and then sent to the second layer in the local server at an affordable communication cost.

In addition, the most popular training mechanism, i.e., the spike-timing-dependent plasticity (STDP), features an unsupervised manner where the training dataset does not need to be labeled. Instead of labeling every data for traditional deep ANNs, the users will only need to label the model one time after the SNN is learned. The concept of STDP is inspired by Hebb’s rule, which means that any two cells or systems of cells that are repeatedly active at the same time will tend to become “associated” so that the activity in one facilitates that in the other [

18]. By implementing the STDP rule, neurons that are excited to one appliance will be associated with the increase of synaptic weights. Therefore, the user only needs to tell the neuron network which neuron excitement pattern corresponds to which appliance. Intuitively, this implies that the user needs to label one data instance for each appliance only.

The contributions of this paper can be summarized as follows.

1) A novel SNN-based NILM algorithm follows the unsupervised learning scheme. The users do not need to label the dataset prior to training.

2) The proposed SNN-based NILM algorithm is especially suitable for low-cost hardware implementation of deep learning.

3) A novel distributed computation mechanism is developed based on SNN to utilize the high-frequency data that are impossible to transmit due to the communication constraint.

The rest of this paper is organized as follows. Section II provides the problem formulation for NILM as well as the background of SNN. In Section III, the novel SNN-based NILM system has been developed along with the novel distributed computation mechanism. Finally, the developed algorithm has been tested using the benchmark public dataset and compared with other state-of-art algorithms in Section IV. Section V concludes the paper.

II. Problem Formulation and Background

In this section, the practical problem of NILM systems is described. Then the SNNs are introduced.

A. Problem Formulation

Given a set of appliances, assume that each appliance is connected with a smart plug which can measure the current and voltage. The smart plug reports to the local household server, and then the local server trains a deep neural network by the unlabeled data. Let $X \in 𝒜 \subseteq R^{2 N \times 2 N}$ denote a feature matrix generated from the voltage and current of an appliance, i.e., $(v, i) \mapsto X : R^{l} \times R^{l} \mapsto 𝒜$ , where v is an l-dimensional voltage vector; i is an l-dimensional current vector; $𝒜$ is the space that contains all feasible X; and $N$ is half of the dimension of the input feature matrix. Consider a feature matrix set ${X_{i} \in 𝒜, 0 < i \leq p}$ along with its corresponding labels $Y_{i} \in ℬ \subseteq Z^{p}$ , where $ℬ$ is the space containing all labels. The input matrices and the output vectors can be written as a set of pairs $X^{'} = {(X_{1}, Y_{1}), (X_{2}, Y_{2}), \dots, (X_{p}, Y_{p})} \in 𝒳$ , where $𝒳$ is the space containing all possible pairs.

Assumption 1: assume there exists a bijection function between space $𝒜$ and space $ℬ$ .

Define a parameter matrix $θ \in R^{n \times m}$ , the cost function $f (X^{'}, θ)$ , and the mapping $𝒳 \mapsto R$ . Also, an optimal parameter $θ^{*}$ is defined as:

θ^{*} = \underset{θ}{a r g m i n} f (X^{'}, θ) \forall X^{'} \in 𝒳

(1)

Next, the classifier operator is defined as follows.

Definition 1: given any member of the vector space $X \in 𝒜$ , let an operator $g (X, θ) : 𝒜 \mapsto ℬ$ satisfy Assumption 1 with the output $Y \in ℬ$ . The operator is called the classification operator, and the corresponding parameter $θ$ is called the weight matrix.

We specifically seek this operator $g (X, θ)$ and its optimal parameter $θ^{*}$ such that the cost function $f ((g (X, θ), Y), θ)$ satisfies (1). With the above definition, we particularly define the cost function as a cross-entropy function, i.e.,

f ((g (X, θ), Y), θ) = - \sum_{Y \in ℬ} p (Y) l n p (g (X, θ))

(2)

where $p (z)$ is the probability of $z$ among all possible $z$ . Intuitively, we want to design a neural network and find its optimal weights so that the input dataset can be correctly classified.

The overall structure of a distributed SNN-based NILM system is demonstrated in Fig. 1, where each household/building includes a set of smart outlets for all appliances. The collected data are prepossessed inside the low-cost smart outlets, and are then sent to the local household server via a shared wireless channel. The local server will continue to learn or classify the received data and upload the type of appliance and down-sampled data to the area server. Finally, the server can suggest the utility companies to adjust the optimal load demand according to the received data of individual appliance. Note that the communication ability among the smart outlets, the local server, and the area server is limited due to the one-to-many style. Moreover, we want to minimize the hardware cost of the smart outlets and the local server for massive deployment.

Fig. 1 Overall structure of a distributed SNN-based NILM system.

B. Biology Inspired SNNs

The information in human brain neurons flows as electric spikes. In a simplified model shown in Fig. 2(a) [

19], [20], the incoming electric signals from the dendrites charge the neuron membrane. After the membrane potential reaches a threshold, the soma emits an action potential. The action potential is a sudden increase (1 ms) in the output voltage, therefore, it is often called spike and modeled by a

δ

-function in most SNN models. After firing, the membrane potential is set to be a reset value, and the neuron enters a short refractory period where it is not excited by any input spikes. This process can be summarized as a function of the membrane potential with respect to the input spikes, as shown in Fig. 2(b). One of the most widely used and biologically plausible spiking neuron models is called the LIF model. The membrane potential increases exponentially when responding to the input spikes. The LIF circuit model is given in Fig. 2(c), where the input spike is replaced by an external current source, and u_th is the firing threshold voltage of the membrane potential. The membrane charging effect is replaced by a capacitor. When the potential of the capacitor exceeds the breakdown voltage, i.e., the threshold potential in SNN, the neuron will emit a spike at the output (not shown in the figure), and the potential of capacitor leaks through the resistor, i.e., reset voltage in SNN. In summary, the membrane potential of an LIF neuron can be written as:

τ_{m} \frac{d u}{d t} = - (u - u_{r e s t}) + R I_{e x t} (t)

(3)

Fig. 2 Structure of a spiking neuron. (a) Biology structure of a spiking neuron. (b) Simplified model of a spiking neuron. (c) Simplified electric circuit for a spiking neuron with LIF model.

where $u$ is the voltage of the membrane potential; $u_{r e s t}$ is the resting voltage of the membrane potential; $τ_{m} = R C$ is a time constant of the neuron; $R$ and $C$ are the resistance and capacitance of the RC circuit representing the neuron, respectively; and $I_{e x t} (t)$ is the external current generated by the input spikes. Moreover, $u = u_{r e s t}$ if $u$ exceeds the firing threshold $u_{t h}$ , and $u_{r e s t}$ is the resting voltage.

III. Design of SNN-based NILM System

A. V-I Trajectory Information Coding

References [

21] and [22] have shown that the performance of NILM algorithm based on the V-I trajectory outperforms the existing algorithms. Similar to [21], the waveform of on/off voltage of each appliance and current are recorded. Then, each voltage and current waveform sequences

v

and

i

are mapped as a binary image [23]. This process is summarized in Algorithm 1. The generated V-I trajectory of some appliances is plotted in Fig. 3. It is easy to observe that different appliances have different V-I trajectories.

Fig. 3 Binary V-I trajectory image of different appliances using data from plug load appliance identification dataset (PLAID). (a) Air conditioner (AC). (b) Compact fluorescent lamp (CFL). (c) Fan. (d) Fridge. (e) Hairdryer. (f) Heater. (g) Incandescent light bulb (ILB). (h) Laptop. (i) Microwave. (j) Vacuum. (k) Washing machine (WM).

Algorithm 1 : generation of V-I binary image
1	Generate the averaged current $i$ and voltage $v$
2	$v_{m a x} \leftarrow m a x (v)$ , $i_{m a x} \leftarrow m a x (i)$ , $v_{m i n} \leftarrow m i n (v)$ , $i_{m i n} \leftarrow m i n (i)$
3	Calculate the average values v₀ and i₀: $v_{0} = \frac{1}{2} (v_{m a x} + v_{m i n})$ , $i_{0} = \frac{1}{2} (i_{m a x} + i_{m i n})$
4	Calculate resolutions $d_{v}$ and $d_{i}$ : $d_{v} = \frac{v_{m a x} - v_{0}}{N}$ , $d_{i} = \frac{i_{m a x} - i_{0}}{N}$
5	Initialize a $2 N \times 2 N$ zero matrix as the image matrix $G$
6	for every data point $(v_{j}, i_{j})$ in $v$ and $i$ do
7	Calculate the coordinate of every data point in the matrix G: $n_{V j} = c e i l (\frac{v_{j} - v_{0}}{d_{v}} + N)$ , $n_{I j} = c e i l (\frac{i_{j} - i_{0}}{d_{i}} + N)$ where $c e i l (\cdot)$ is the ceiling function; and $n_{V j}$ and $n_{I j}$ are the coordinates of the voltage axis and current axis, respectively
8	if $0 < n_{V j} < 2 N + 1$ and $0 < n_{I j} < 2 N + 1$ then
9	$G (n_{V j}, n_{I j}) \leftarrow 1$
10	end if
11	end for

Next, the binary V-I trajectory image is encoded to the spikes. In biology, the timing of action potentials (spikes) is highly irregular [

24], which suggests a random process to encode the analog input signal. The most commonly used method is the Poisson spike generation [25], where the analog signal is utilized as the rate of a spike train which follows Poisson distribution. The probability of the spike number follows Poisson distribution and the expected value is proportional to the input intensity. With higher analog input, more spikes are generated to the input layer. The process of encoding a V-I trajectory image into spikes is shown in Fig. 4. To input the 2D V-I trajectory image, the picture is first flattened to 1D, and then each entry of the array is used as the analog input of the corresponding input neurons to generate the Poisson spike trains. In Fig. 4, the black and white squares are the pixels of the image; the blue circles are neurons; and the output voltage spikes of the input neurons are plotted on the right-hand side of the figure.

Fig. 4 Process of encoding a V-I trajectory image into spikes.

B. Neurons and STDP Learning Model

Instead of the general LIF model shown in (3), this paper divides the external input into two types, i.e., the excitatory and inhibitory parts for the best performance on the classification tasks [

26]. As a consequence, the update process of excitatory and inhibitory synapses also follows different parameters.

The membrane potential is calculated as:

τ_{m} \frac{d u}{d t} = (u_{r e s t} - u) + g_{e} (u_{e} - u) + g_{i} (u_{i} - u)

(4)

where $u_{i}$ and $u_{e}$ are the equilibrium potentials of inhibitory and excitatory synapse [

26], respectively; and

g_{e}

and

g_{i}

are the synaptic conductances of the excitatory and inhibitory components, respectively.

Compared with the original LIF model (3), the $R C c i r c u i t$ term and the external current $I_{e x t} (t)$ are replaced by two separate synapses, which creates the competition between neurons to improve the overall performance [

26].

The synaptic conductance is, however, not known if we seek the optimal performance. The process of finding better synaptic conductance, which is related to the synaptic weights, are usually named as learning in traditional ANNs. Different from ANNs where the weights are learned in a supervised manner using gradient descent, the SNNs perform well in the unsupervised learning method known as the STDP. As the key component of the SNN, the conductance decays exponentially unless a presynaptic spike arrives.

τ_{e} \frac{d g_{e}}{d t} = - g_{e}

(5)

where $τ_{e}$ is a time constant. Let $w_{i j}$ be the synaptic weight between the neurons $i$ and $j$ . The conductance increases $w_{i j}$ for each presynaptic spike. The inhibitory conductance $g_{e}$ updates in the same fashion but with a different time constant $τ_{i}$ . Based on (5), if a neuron $i$ receives presynaptic spikes from neuron $j$ on a more frequent basis, the conductance $g_{e i j}$ would increase, which means that more frequent firing behaviors will appear in neurons $i$ and $j$ with similar input. Intuitively, the synaptic weights between these neurons are greater to react to a certain input pattern. To tune the evolution of the conductance, a learning rule for the weight change $Δ w$ is proposed in [

26], i.e.,

Δ w = α (x_{p r e} - x_{t a r}) (w_{m a x} {- w)}^{μ}

(6)

where $α$ is the learning rate; $w_{m a x}$ is the maximum weight; $x_{t a r}$ is a offset value; $μ$ is an eligibility trace which denotes the dependence of the previous weight; and $x_{p r e}$ is the presynaptic trace parameter, which contains the information of all presynaptic spikes. The presynaptic trace is increased by 1 for every arrived presynaptic spike and decays exponentially otherwise. When the presynaptic trace is larger than the offset value of $x_{p r e}$ , the weight of this synapse is enhanced. Therefore, the conductance is increased by a greater value. However, when $x_{p r e}$ does not reach the offset value, the conductance of this synapse will be reduced. This mechanism along with the conductance adjustment ensures that the neurons responding strongly to one input get more connected while irrelevant neurons are disconnected by decreasing the conductance.

Unlike the back-propagation method, which is widely used in recent deep ANN learning rules, the STDP rule can adjust the synapse of neurons without a reference output guidance. In a trained SNN, a specific neuron set is excited by a corresponding type of appliance. The users are only required to label one data instance for each type of appliance to associate the neuron excitement patterns (output spike rates) with appliance types. This relieves the assumption of a large set of labeled data, which makes it more reasonable to commercialize the NILM system.

To this end, the SNN for appliance classification along with the unsupervised learning rules has been introduced. Next, a distributed NILM system is proposed with the SNN embedded.

C. SNN-based NILM System Design

As mentioned above, the NILM system hardware follows a three-layer structure, i.e., the smart outlet, the local household server, and the area server. In practical applications such as the Grid Sense system, the communication of the smart outlets and the local server is often limited due to the requirement of low-cost hardware. This stringent constraint frustrates many existing deep learning methods such as [

5], [27]. However, the biology-inspired spiking neurons can be constructed by a simple RC circuit shown in Fig. 2. The information flows in the form of binary electric pulses instead of the analog signal. In traditional NILM designs such as [28], the deep neural networks are trained and stored in the local server, while the smart outlet is used to collect and transmit data only. Due to the communication limit, this setup suffers from the data sampling rate, which means that the data with high sampling rate cannot be transmitted to the server in real time. To deal with this issue, the SNN is designed in a novel distributed computation fashion, where each of the server and the smart outlet has a part of the SNN. In this design, the data is collected by the analog/digital (AD) module in the smart outlet, and then directly processed by the input layer. Instead of the analog data, the outputs of the smart outlet are sparse binary pulses that can be easily compressed without distortion.

The structure of the proposed NILM system is shown in Fig. 5. The smart outlet collects the instant voltage and current data at a high sampling rate. Then, the data cluster in a fixed time window is preprocessed into a binary V-I trajectory picture. To input it into the SNN, the image is flattened into an array and sent to the neurons in the input layer. The input neurons generate Poisson spike trains based on the rate of each pixel. Note that the spike trains are essentially a binary spike vectors that can be compressed into a small-timestamp sequence of the spike time. Therefore, the compressed sequence of spike time is transmitted into the hidden and output layers located in the local server. The channel between the input layer and the hidden layer is wireless. Since the transmitted data amount is small, the communication ability between the smart outlet and the local server is low.

Fig. 5 Structure of NILM system.

The received sequence of spike time is then decoded into spike trains and sent to the connected neurons in the hidden layer in the local server. The connections between the hidden and output layers are the circuits, and the information flows as electric pulses. Compared with the traditional ANN with various activation functions that requires an intensive computation unit, the SNN neurons are physical RC circuit, and the updated law does not require solving the differential equation. Therefore, the computation efforts of both learning and classification are greatly reduced. The speed of the forward process of electric flow is also much higher than that of digital computation. Consequently, the costs of the smart outlet and the household server are reduced, because no high-end computation-intensive unit such as the GPU is required.

The training process is represented as a pseudo-code in Algorithm 2. Note that the user only needs to label one time for each appliance. And then, this model can be used in all trained appliances.

In summary, the SNN-based NILM algorithm can improve the traditional appliance inference in four aspects: ① the cost of hardware is reduced; ② the communication requirement is lowered; ③ the running speed is increased; and ④ the need for numerous labeled data is canceled.

Algorithm 2 : training process of SNN-based NILM algorithm
1	if appliance is turned on then
2	Measure the current and voltage and maintain a buffer at $w_{z}$ size
3	if buffer is full then
4	Pop the first current and voltage measurement
5	Generate the V-I trajectory using Algorithm 1
6	Flatten the V-I image and input the SNN
7	Update all weights according to STDP (6)
8	if weights are updated for $K$ times then
9	Break
10	end if
11	Measure the current and voltage
12	end if
13	end if
14	Let the user label this appliance

IV. Experiments and Results

In this section, an SNN with three layers is constructed and then tested using a public dataset.

A. Data Preparation

We utilize the benchmark public PLAID dataset [

29] to test the performance of the proposed SNN-based NILM algorithm. The PLAID dataset has 11 appliances which include 1094 instances. The details are listed in Table I. Each instance contains 1 s of recorded voltage and current pairs at a 30 kHz sampling rate.

Table I Summery of PLAID Dataset

Appliance type	Appliance number	Instance
AC	14	66
CFL	31	175
Fan	23	115
Fridge	21	38
Hairdryer	32	156
Heater	7	35
ILB	23	114
Laptop	34	172
Microwave	28	139
Vacuum	8	38
WM	9	26

To reduce the input complexity of SNN, each instance is uniformly downsampled to the size of $28 \times 28$ . Then, the V-I trajectory image is generated using the method described in Section II. We randomly select $80 %$ of the data as the training data and the rest as testing data. To this end, the training and testing data are prepared. It is worth noting that the label of these appliances is not required, which complies with the unsupervised training method.

B. SNN Structure

The SNN includes four layers, i.e., input layer, two hidden layers, and output layer. The input layer includes 784 ( $28 \times 28$ ) coding neurons, which are the Poisson distribution generators. Each pixel of the binary V-I trajectory image is connected with one input neuron. There are 400 ( $20 \times 20$ ) neurons in each of the two hidden layers and 11 output neurons in the output layer. The synapses between the input and hidden layers and the hidden and output layers follow the “all-to-all” fashion, meaning that all neurons are connected. The proposed SNN structure is shown in Fig. 6 and the layers of the proposed SNN are shown in Table II.

Fig. 6 Proposed SNN structure.

Table II Layers of Proposed SNN

Layer type	Activation function	Size	Number of neurons	Data preprocess
Input layer	LIF spike generator	28×28	784	min-max normalization
Hidden layer 1	LIF spike generator	20×20	400
Hidden layer 2	LIF spike generator	20×20	400
Output layer	Summation and maximum	1×11	11

The resting voltages $u_{r e s t}$ are set to be $- 65$ $m V$ to 65 mV and $- 60$ $m V$ to 60 mV for the two hidden layers, respectively. The membrane potential thresholds $u_{t h}$ are set to be $- 52$ $m V$ to 52 mV and $- 40$ $m V$ to 40 mV for the two hidden layers, respectively. The Brian [

30] library is utilized to code the proposed SNN.

C. Training and Testing Results

We set the time for a single simulation to be 0.35 s. Note that this time represents the STDP learning method performed for 0.35 s for each single V-I trajectory image, $i . e .$ , instance. And the total simulation time depends on the number of instances in the training set. As previously mentioned, we set $80 %$ of the data as the training data, which results in $860$ instances. To increase the prediction accuracy of SNN, we have repeated the $860$ instances several times and then shuffled the repeated training data. When all instances in the training dataset are iterated to update the synaptic weights and conductance, we input the only one labeled data instance for each appliance to observe the neurons’ behavior. Each neuron in the output layer is labeled with the most excited appliance type, i.e., with the highest spikes rate. For example, when the 11 labeled data are input into the SNN, the first neuron in the output layer finds that AC has the highest output spikes rate, then this neuron is marked as AC type. In the testing, when an unknown instance is regarded as an input into the SNN, the output spike rates of the neurons of the same type are averaged. Then the neuron type with the highest average spike rate is selected as the classified appliance type.

The testing data contains $20 %$ of the whole PLAID dataset. The overall accuracy Acc is computed by:

A c c = \frac{N_{c o r r e c t}}{N_{t o t a l}}

(7)

where N_correct and N_total are the numbers of correct classification and total classification, respectively.

As a result, the classification of the test data reports an overall accuracy of $83 %$ . The confusion matrix of the test data is given in Fig. 7, where the row shows the real types and the column shows the classified types. The two sub-plots on the right and at the bottom represent the percentages of correctly and incorrectly classified observations for each true class, respectively. The results show that the SNN performs well on most appliances but poorly on the hairdryer and AC. A more detailed calculation of accuracy for each type of appliance is illustrated in Fig. 8. Due to the nature of unsupervised STDP on SNN, the learned patterns are encoded in the intensity of the synaptic weights [

31], [32]. Therefore, to demonstrate that the appliance features have been learned, the scale of the weights between the input layer and the first hidden layer are shown in Fig. 9, where the lighter and darker yellow colors represent lower and higher values of the entry of the weight matrix, respectively. For each neuron in the first hidden layer, 784 (

28 \times 28

) input neurons are connected. Therefore, the number of synapses between one hidden neuron and all input neurons is 784. We plot them as

28 \times 28

matrix for better visualization. Moreover, all synapses of the 400 hidden neurons are arranged as a

20 \times 20

image matrix where each cell contains a

28 \times 28

matrix, i.e., the synapses of one hidden neuron. In Fig. 9, we can see the image patterns of different types of appliances. The original training dataset from PLAID only includes

860

instances. Therefore, we repeatedly sample instances from the original dataset to provide enough training data. The evolution of classification results with respect to the training data size is plotted in Fig. 10. It is observed that the accuracy increases as the training set size is enlarged.

Fig. 7 Confusion matrix of test results.

Fig. 8 Computation of accuracy for each type of appliance.

Fig. 9 Synaptic weight matrix of synapses between input layer and hidden layer with each synaptic weights’ pattern matrix having learned V-I trajectory of one type of appliance.

Fig. 10 Accuracy plot with respect to training dataset size.

Next, the results are analyzed using the benchmark evaluation indices and then compared with traditional ANN methods. According to [

33], the precision, recall, and F-score are used to compare with other algorithms, i.e.,

F_S c o r e = 2 \frac{P r e c i s i o n \cdot R e c a l l}{P r e c i s i o n + R e c a l l}

(8)

P r e c i s i o n = \frac{T P}{T P + F P}

(9)

R e c a l l = \frac{T P}{T P + F N}

(10)

where TP, TN, FP, FN represent the true positive, true negative, false positive, and false negative classifications, respectively.

The precision, recall, and F-score are computed and listed in Table III. Moreover, the F1-score of each appliance is compared with a popular ANN-based NILM algorithm, i.e., convolution neural network (CNN) [

33]. The comparison is shown in Fig. 11, where the proposed SNN outperforms the CNN in most appliances. The average F1-score of SNN and CNN are 0.7942 and 0.7761, respectively. In addition, we compare the overall accuracy of the proposed SNN with both the CNN method and the traditional random forest algorithm in Table IV.

Table III Precision, Recall, and F-score of SNN

Appliance type	Precision	Recall	F-score
AC	0.41	1.00	0.59
CFL	1.00	0.85	0.92
Fan	0.92	0.50	0.65
Fridge	1.00	0.71	0.83
Hairdryer	1.00	0.48	0.65
Heater	0.40	1.00	0.57
ILB	0.81	0.95	0.88
Laptop	0.88	0.88	0.88
Microwave	1.00	1.00	1.00
Vacuum	1.00	1.00	1.00
WM	0.63	1.00	0.77

Fig. 11 Average F1-score of SNN and CNN.

Table IV Overall Accuracy Comparison

Reference	Algorithm	Overall accuracy
[33]	CNN	0.82
[34]	Random forest	0.78
Proposed	SNN	0.83

Finally, we compare the performance of the proposed algorithm with that of the transfer learning (TL) algorithm [

35], which also claims to reduce the labeled training set size to generate a new model. Different from the proposed unsupervised method, [35] follows the supervised learning fashion, where most layers of the neural network comes from a pretrained deep CNN named the AlexNet [12]. Thus, the algorithm in [35] requires less labeled data since only a few layers need to be retrained. After testing with the same PLAID dataset, the overall accuracy is presented in Table V. We can observe that the AlexNet-TL algorithm performs better than the proposed algorithm when the labeled data are enough. However, in the case where almost no labeled data can be obtained, the proposed algorithm significantly outperforms the AlexNet-TL method.

Table V Accuracy Comparison with TL Algorithm

Algorithm	Labeled training data	Overall accuracy
AlexNet-TL [35]	1133	0.99
AlexNet-TL [35]	0	0.24
Proposed	1	0.83

Combined with the previous analysis, it can be observed that the SNN method not only outperforms the traditional ANN in classification accuracy of most appliances, but is especially useful in practical NILM scenarios where very few labeled data are provided. Moreover, it is known that the SNN is faster and can be implemented with low-cost electronic components easily.

V. Discussion and Conclusion

In this paper, a novel SNN-based NILM algorithm is proposed along with the distributed computation fashion to better embed the NILM functionality in a practical load demand analysis system, i.e., the Grid Sense system. The proposed NILM algorithm utilizes the biology-inspired SNN and features an unsupervised learning scheme named STDP. To learn a new household model, the user only needs to label one data instance for each appliance, which costs significantly less effort than training the traditional ANNs. Moreover, the nature of SNN makes it extremely easy to be implemented into low-cost hardware where an RC circuit can represent each neuron. With this setup, the expensive computational intensive unit, $i . e .$ , GPU, is no longer needed for smart outlets so that the massive deployment in households becomes possible. Another advantage caused by the hardware friendly representation of the SNN is the distributed computation scheme. Instead of the digitized analog signal in most ANNs, the signal of SNN flows as a sparse binary vector, i.e., the spike train. As a result, the SNN layers can be easily separated into different devices to access high sampling rate measurements without powerful communication equipment. The experiments are conducted on the PLAID dataset to show that the proposed SNN method can accurately classify the appliances. The performance is better than traditional CNN and other algorithms. Specifically, the proposed unsupervised SNN algorithms have been compared with the other popular deep-learning-based algorithms such as the AlexNet-TL [

21] and CNN [33]. The results show that the accuracy of the proposed SNN algorithm reaches

83 %

while that of CNN is

82 %

. In the sense of NILM, the proposed unsupervised learning method has similar performance to traditional deep-learning-based algorithms. In the deep-learning-based algorithm that also aims to reduce the size of training data [21], the accuracy of the proposed SNN algorithm is

83 %

. In contrast, the accuracy in [21] is

24 %

if we restrict the number of the labeled data to one. However, [21] has reported that by using colored V-I trajectory, the performance will be improved significantly. Therefore, we will develop an SNN-based method using the colored V-I trajectory, and then apply it to the Grid Sense system.

REFERENCES

Y. Kabalci, “A survey on smart metering and smart grid communication,” Renewable and Sustainable Energy Reviews, vol. 57, pp. 302-318, May 2016. [Baidu Scholar]

J. Duan, C. Wang, H. Xu et al., “Distributed control of inverter-interfaced microgrids based on consensus algorithm with improved transient performance,” IEEE Transactions on Smart Grid, vol. 10, no. 2, pp. 1303-1312, Mar. 2017. [Baidu Scholar]

Z. Zhou, Y Xiang, H. Xu et al., “Self-organizing probability neural network based intelligent non-intrusive load monitoring with applications to low-cost residential measuring devices,” Transactions of the Institute of Measurement and Control, vol. 43, no. 3, pp. 635-645, Sept. 2020. [Baidu Scholar]

A. S. Bouhouras, P. A. Gkaidatzis, K. C. Chatzisavvas et al., “Load signature formulation for non-intrusive load monitoring based on current measurements,” Energies, vol. 10, no. 4, p. 538, Apr. 2017. [Baidu Scholar]

J. Kelly and W. Knottenbelt, “Neural NILM: deep neural networks applied to energy disaggregation,” in Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-efficient Built Environments, Seoul, South Korea, Nov. 2015, pp. 55-64. [Baidu Scholar]

S. Makonin, F. Popowich, I. V. Bajić et al., “Exploiting hmm sparsity to perform online real-time nonintrusive load monitoring,” IEEE Transactions on Smart Grid, vol. 7, no. 6, pp. 2575-2585, Nov. 2015. [Baidu Scholar]

N. Batra, A. Singh, and K. Whitehouse. (2015, Oct.). Neighbourhood NILM: a big-data approach to household energy disaggregation. [Online]. Available: https://arxiv.org/abs/1511.02900v1 [Baidu Scholar]

O. Krystalakos, C. Nalmpantis, and D. Vrakas, “Sliding window approach for online energy disaggregation using artificial neural networks,” in Proceedings of the 10th Hellenic Conference on Artificial Intelligence, Patras, Greece, Jul. 2018, pp. 1-6. [Baidu Scholar]

J. M. Gillis, S. M. Alshareef, and W. G. Morsi, “Nonintrusive load monitoring using wavelet design and machine learning,” IEEE Transactions on Smart Grid, vol. 7, no. 1, pp. 320-328, Jan. 2015. [Baidu Scholar]

Z. Zhou, Y. Xiang, H. Xu et al., “A novel transfer learning-based intelligent nonintrusive load-monitoring with limited measurements,” IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-10, Jul. 2020. [Baidu Scholar]

Y. Xiang, X. Lu, Z. Yu et al., “IoT and edge computing based direct load control for fast adaptive frequency regulation,” in Proceedings of IEEE PES General Meeting, Atlanta, USA, Aug. 2019, pp. 1-5. [Baidu Scholar]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Proceedings of the Advances in Neural Information Processing Systems, Reno, USA, Dec. 2012, pp. 1097-1105. [Baidu Scholar]

K. Ehsani, H. Bagherinezhad, J. Redmon et al., “Who let the dogs out? Modeling dog behavior from visual data,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, Jun. 2018, pp. 4051-4060. [Baidu Scholar]

H. Jahangir, H. Tayarani, S. Baghali et al., “A novel electricity price forecasting approach based on dimension reduction strategy and rough artificial neural networks,” IEEE Transactions on Industrial Informatics, vol. 16, no. 4, pp. 2369-2381, Apr. 2019. [Baidu Scholar]

M. Bouvier, A. Valentian, T. Mesquida et al., “Spiking neural networks hardware implementations and challenges: a survey,” ACM Journal on Emerging Technologies in Computing Systems (JETC), vol. 15, no. 2, pp. 1-35, Apr. 2019. [Baidu Scholar]

S. Dutta, V. Kumar, A. Shukla et al., “Leaky integrate and fire neuron by charge-discharge dynamics in floating-body mosfet,” Scientific Reports, vol. 7, no. 1, pp. 1-7, Aug. 2017. [Baidu Scholar]

B. Das, J. Schulze, and U. Ganguly, “Ultra-low energy LIF neuron using Si NIPIN diode for spiking neural networks,” IEEE Electron Device Letters, vol. 39, no. 12, pp. 1832-1835, Dec. 2018. [Baidu Scholar]

J. L. Lobo, J. Del Ser, A. Bifet et al., “Spiking neural networks and online learning: an overview and perspectives,” Neural Networks, vol. 121, pp. 88-100, Sept. 2019. [Baidu Scholar]

Z. Bing, C. Meschede, F. Röhrbein et al., “A survey of robotics control based on learning-inspired spiking neural networks,” Frontiers in Neurorobotics, vol. 12, p. 35, Jul. 2018. [Baidu Scholar]

Wikipedia. (2020, Apr.). Neuron. [Online]. Available: https://en.wikipedia.org/wiki/Neuron/ [Baidu Scholar]

Y. Liu, X. Wang, and W. You, “Non-intrusive load monitoring by voltage-current trajectory enabled transfer learning,” IEEE Transactions on Smart Grid, vol. 10, no. 5, pp. 5609-5619, Dec. 2018. [Baidu Scholar]

D. F. Teshome, T. Huang, and K.-L. Lian, “Distinctive load feature extraction based on Fryze’s time-domain power theory,” IEEE Power and Energy Technology Systems Journal, vol. 3, no. 2, pp. 60-70, Apr. 2016. [Baidu Scholar]

L. Du, D. He, R. G. Harley et al., “Electric load classification by binary voltage-current trajectory mapping,” IEEE Transactions on Smart Grid, vol. 7, no. 1, pp. 358-365, Jun. 2015. [Baidu Scholar]

W. Gerstner, W. M. Kistler, R. Naud et al., Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge: Cambridge University Press, 2014. [Baidu Scholar]

J. Hahne, D. Dahmen, J. Schuecker et al., “Integration of continuous-time dynamics in a spiking neural network simulator,” Frontiers in Neuroinformatics, vol. 11, p. 34, May 2017. [Baidu Scholar]

P. U. Diehl and M. Cook, “Unsupervised learning of digit recognition using spike-timing-dependent plasticity,” Frontiers in Computational Neuroscience, vol. 9, p. 99, Aug. 2015. [Baidu Scholar]

W. Kong, Z. Y. Dong, B. Wang et al., “A practical solution for non-intrusive type II load monitoring based on deep learning and post-processing,” IEEE Transactions on Smart Grid, vol. 11, no. 1, pp. 148-160, May 2019. [Baidu Scholar]

S. Barker, M. Musthag, D. Irwin et al., “Non-intrusive load identification for smart outlets,” in Proceedings of 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm), Venice, Italy, Nov. 2014, pp. 548-553. [Baidu Scholar]

J. Gao, S. Giri, E. C. Kara et al., “PLAID: a public dataset of high-resolution electrical appliance measurements for load identification research: demo abstract,” in Proceedings of the 1st ACM Conference on Embedded Systems for Energy-efficient Buildings, New York, USA, Nov. 2014, pp. 198-199. [Baidu Scholar]

M. Stimberg, R. Brette, and D. F. Goodman, “Brian 2, an intuitive and efficient neural simulator,” eLife, vol. 8, p. e47314, Aug. 2019. [Baidu Scholar]

T. Iakymchuk, A. Rosado-Muñoz, J. F. Guerrero-Martinez et al., “Simplified spiking neural network architecture and STDP learning algorithm applied to image classification,” EURASIP Journal on Image and Video Processing, vol. 2015, no. 1, p. 4, Feb. 2015. [Baidu Scholar]

M. Mozafari, S. R. Kheradpisheh, T. Masquelier et al., “First-spike-based visual categorization using reward-modulated STDP,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 12, pp. 6178-6190, May 2018. [Baidu Scholar]

L. de Baets, J. Ruyssinck, C. Develder et al., “Appliance classification using VI trajectories and convolutional neural networks,” Energy and Buildings, vol. 158, pp. 32-36, Jan. 2018. [Baidu Scholar]

J. Gao, E. C. Kara, S. Giri et al., “A feasibility study of automated plug-load identification from high-frequency measurements,” in Proceedings of 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, USA, Dec. 2015, pp. 220-224. [Baidu Scholar]

Address:No.19 Chengxin Avenue, Jiangning District, Nanjing 211106, China

E-mail: mpce@alljournals.cn

Tel:86-25-81093060

Fax:86-25-81093040

Home

Introduction

Editorial Board

For Author

Call For Papers

APC

Sponsor & Publisher