0
$\begingroup$

I have the a data set representing the electricity consumption of 25 000 customer. The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from counters of residential, commercial and Industrial buildings thing that makes us have different consumption modes. Each record in the dataset is in the format:

meter_id timestamp cumulative_value

using pandas function groupby('meter_id') and by transforming the each group to an array format I get the array format that corresponds to an independent time series ready to feed to a keras LSTM model.

The problem that I have is that my model does not converge no matter the learning rate is. I tried all kind of normalization and standardization but still having this over-fitting problem. Displaying the probability distribution of all the consumption values is as follows: Histogram of data consumption for for all smart meters

Here is also the histogram after normalization:

enter image description here

My LSTM network configuration is as follows:

model_cum = Sequential()
model_cum.add(LSTM(units=50,return_sequences=True,input_shape=(xt.shape[1],1))) # for multiple series 
model_cum.add(Dropout(0.2))
model_cum.add(LSTM(units= 50,return_sequences=True ))
model_cum.add(Dropout(0.2))
model_cum.add(LSTM(units=50, return_sequences=False))
model_cum.add(Dropout(0.2))
model_cum.add(Dense(1))

and the output I get is the following:

Train on 15281 samples, validate on 1698 samples
Epoch 1/20
15281/15281 [==============================] - 118s 8ms/sample - loss: 0.4545 - mean_squared_error: 0.2075 - val_loss: 0.4467 - val_mean_squared_error: 0.1998
Epoch 2/20
15281/15281 [==============================] - 117s 8ms/sample - loss: 0.4378 - mean_squared_error: 0.1926 - val_loss: 0.4301 - val_mean_squared_error: 0.1852
Epoch 3/20
15281/15281 [==============================] - 117s 8ms/sample - loss: 0.4213 - mean_squared_error: 0.1784 - val_loss: 0.4134 - val_mean_squared_error: 0.1711
Epoch 4/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.4045 - mean_squared_error: 0.1646 - val_loss: 0.3966 - val_mean_squared_error: 0.1575
Epoch 5/20
15281/15281 [==============================] - 117s 8ms/sample - loss: 0.3876 - mean_squared_error: 0.1512 - val_loss: 0.3797 - val_mean_squared_error: 0.1444
Epoch 6/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.3706 - mean_squared_error: 0.1384 - val_loss: 0.3626 - val_mean_squared_error: 0.1317
Epoch 7/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.3535 - mean_squared_error: 0.1260 - val_loss: 0.3453 - val_mean_squared_error: 0.1194
Epoch 8/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.3362 - mean_squared_error: 0.1141 - val_loss: 0.3277 - val_mean_squared_error: 0.1076
Epoch 9/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.3183 - mean_squared_error: 0.1024 - val_loss: 0.3098 - val_mean_squared_error: 0.0962
Epoch 10/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.3002 - mean_squared_error: 0.0912 - val_loss: 0.2916 - val_mean_squared_error: 0.0852
Epoch 11/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.2818 - mean_squared_error: 0.0805 - val_loss: 0.2730 - val_mean_squared_error: 0.0747
Epoch 12/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.2635 - mean_squared_error: 0.0706 - val_loss: 0.2540 - val_mean_squared_error: 0.0647
Epoch 13/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.2441 - mean_squared_error: 0.0608 - val_loss: 0.2346 - val_mean_squared_error: 0.0553
Epoch 14/20
15281/15281 [==============================] - 114s 7ms/sample - loss: 0.2243 - mean_squared_error: 0.0516 - val_loss: 0.2147 - val_mean_squared_error: 0.0463
Epoch 15/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.2037 - mean_squared_error: 0.0428 - val_loss: 0.1943 - val_mean_squared_error: 0.0380
Epoch 16/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.1832 - mean_squared_error: 0.0349 - val_loss: 0.1731 - val_mean_squared_error: 0.0302
Epoch 17/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.1624 - mean_squared_error: 0.0278 - val_loss: 0.1513 - val_mean_squared_error: 0.0231
Epoch 18/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.1396 - mean_squared_error: 0.0210 - val_loss: 0.1287 - val_mean_squared_error: 0.0168
Epoch 19/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.1168 - mean_squared_error: 0.0152 - val_loss: 0.1051 - val_mean_squared_error: 0.0113
Epoch 20/20
15281/15281 [==============================] - 114s 7ms/sample - loss: 0.0932 - mean_squared_error: 0.0103 - val_loss: 0.0808 - val_mean_squared_error: 0.0067

There mean_square error loss in this case is always under zero and nearly fixed. What is the problem with what I am doing and how to solve it ? enter image description here

$\endgroup$
1
$\begingroup$

You could try using the box cox transformation on your target variable to make it normal-distributed, and repeat the same process.

Then you will have to transform the values of the predictions with the inverse transformation . The two functions are on scipy.

from scipy.special import  inv_boxcox
from scipy.stats import boxcox
$\endgroup$
  • $\begingroup$ In case this works, how to denormalize back the data to make predictions? $\endgroup$ – pentanol Sep 24 at 13:00
  • $\begingroup$ @pentanol with the inverse transformation ? $\endgroup$ – Duccio Piovani Sep 24 at 13:03
  • $\begingroup$ Yes, I mean what is the inverse of box cox transfrom $\endgroup$ – pentanol Sep 24 at 13:07
  • $\begingroup$ docs.scipy.org/doc/scipy/reference/generated/… I will edit the answer $\endgroup$ – Duccio Piovani Sep 24 at 13:11
  • $\begingroup$ The idea was good, the the dataset has the shape of of normal distribution but I still suffer from underfitting as mentioned above. $\endgroup$ – pentanol Sep 24 at 13:59
0
$\begingroup$

The solution is simple. The model is underfitting, that means it cant learn from the data. The solution is to increase the number of LSTM layers and increase the Dropout to avoid overfitting. For more details you can see the following link

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.