0
$\begingroup$

I have the a data set representing the electricity consumption of 25 000 customer. The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from counters of residential, commercial and Industrial buildings thing that makes us have different consumption modes. Each record in the dataset is in the format: 

meter_id timestamp cumulative_value

using pandas function groupby('meter_id') and by transforming the each group to an array format I get the array format that corresponds to an independent time series ready to feed to a keras LSTM model.

The problem that I have is that my model does not converge no matter the learning rate is. I tried all kind of normalization and standardization but still having this over-fitting problem. Displaying the probability distribution of all the consumption values is as follows: Histogram of data consumption for for all smart meters

Here is also the histogram after normalization:

enter image description here

My LSTM network configuration is as follows: 

model_cum = Sequential()
model_cum.add(LSTM(units=50,return_sequences=True,input_shape=(xt.shape[1],1))) # for multiple series 
model_cum.add(Dropout(0.2))
model_cum.add(LSTM(units= 50,return_sequences=True ))
model_cum.add(Dropout(0.2))
model_cum.add(LSTM(units=50, return_sequences=False))
model_cum.add(Dropout(0.2))
model_cum.add(Dense(1))

and the output I get is the following: 

Train on 15281 samples, validate on 1698 samples
Epoch 1/20
15281/15281 [==============================] - 118s 8ms/sample - loss: 0.4545 - mean_squared_error: 0.2075 - val_loss: 0.4467 - val_mean_squared_error: 0.1998
Epoch 2/20
15281/15281 [==============================] - 117s 8ms/sample - loss: 0.4378 - mean_squared_error: 0.1926 - val_loss: 0.4301 - val_mean_squared_error: 0.1852
Epoch 3/20
15281/15281 [==============================] - 117s 8ms/sample - loss: 0.4213 - mean_squared_error: 0.1784 - val_loss: 0.4134 - val_mean_squared_error: 0.1711
Epoch 4/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.4045 - mean_squared_error: 0.1646 - val_loss: 0.3966 - val_mean_squared_error: 0.1575
Epoch 5/20
15281/15281 [==============================] - 117s 8ms/sample - loss: 0.3876 - mean_squared_error: 0.1512 - val_loss: 0.3797 - val_mean_squared_error: 0.1444
Epoch 6/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.3706 - mean_squared_error: 0.1384 - val_loss: 0.3626 - val_mean_squared_error: 0.1317
Epoch 7/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.3535 - mean_squared_error: 0.1260 - val_loss: 0.3453 - val_mean_squared_error: 0.1194
Epoch 8/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.3362 - mean_squared_error: 0.1141 - val_loss: 0.3277 - val_mean_squared_error: 0.1076
Epoch 9/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.3183 - mean_squared_error: 0.1024 - val_loss: 0.3098 - val_mean_squared_error: 0.0962
Epoch 10/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.3002 - mean_squared_error: 0.0912 - val_loss: 0.2916 - val_mean_squared_error: 0.0852
Epoch 11/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.2818 - mean_squared_error: 0.0805 - val_loss: 0.2730 - val_mean_squared_error: 0.0747
Epoch 12/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.2635 - mean_squared_error: 0.0706 - val_loss: 0.2540 - val_mean_squared_error: 0.0647
Epoch 13/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.2441 - mean_squared_error: 0.0608 - val_loss: 0.2346 - val_mean_squared_error: 0.0553
Epoch 14/20
15281/15281 [==============================] - 114s 7ms/sample - loss: 0.2243 - mean_squared_error: 0.0516 - val_loss: 0.2147 - val_mean_squared_error: 0.0463
Epoch 15/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.2037 - mean_squared_error: 0.0428 - val_loss: 0.1943 - val_mean_squared_error: 0.0380
Epoch 16/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.1832 - mean_squared_error: 0.0349 - val_loss: 0.1731 - val_mean_squared_error: 0.0302
Epoch 17/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.1624 - mean_squared_error: 0.0278 - val_loss: 0.1513 - val_mean_squared_error: 0.0231
Epoch 18/20
15281/15281 [==============================] - 116s 8ms/sample - loss: 0.1396 - mean_squared_error: 0.0210 - val_loss: 0.1287 - val_mean_squared_error: 0.0168
Epoch 19/20
15281/15281 [==============================] - 115s 8ms/sample - loss: 0.1168 - mean_squared_error: 0.0152 - val_loss: 0.1051 - val_mean_squared_error: 0.0113
Epoch 20/20
15281/15281 [==============================] - 114s 7ms/sample - loss: 0.0932 - mean_squared_error: 0.0103 - val_loss: 0.0808 - val_mean_squared_error: 0.0067

There mean_square error loss in this case is always under zero and nearly fixed. What is the problem with what I am doing and how to solve it ? enter image description here

Improve this question
$\endgroup$

3 Answers 3

Reset to default
1
$\begingroup$

You could try using the box cox transformation on your target variable to make it normal-distributed, and repeat the same process.

Then you will have to transform the values of the predictions with the inverse transformation . The two functions are on scipy. 

from scipy.special import  inv_boxcox
from scipy.stats import boxcox
Improve this answer
$\endgroup$
5
  • $\begingroup$ In case this works, how to denormalize back the data to make predictions? $\endgroup$
    – pentanol
    Sep 24, 2019 at 13:00
  • $\begingroup$ @pentanol with the inverse transformation ? $\endgroup$
    – Duccio Piovani
    Sep 24, 2019 at 13:03
  • $\begingroup$ Yes, I mean what is the inverse of box cox transfrom $\endgroup$
    – pentanol
    Sep 24, 2019 at 13:07
  • $\begingroup$ docs.scipy.org/doc/scipy/reference/generated/… I will edit the answer $\endgroup$
    – Duccio Piovani
    Sep 24, 2019 at 13:11
  • $\begingroup$ The idea was good, the the dataset has the shape of of normal distribution but I still suffer from underfitting as mentioned above. $\endgroup$
    – pentanol
    Sep 24, 2019 at 13:59
0
$\begingroup$

The solution is simple. The model is underfitting, that means it cant learn from the data. The solution is to increase the number of LSTM layers and increase the Dropout to avoid overfitting. For more details you can see the following link

Improve this answer
$\endgroup$
0
$\begingroup$

First of all, are you sure you are using a MSE loss? If so, the loss should go hand in hand with your metric (MSE), especially since I see no Masking layer in your network. Plus, the MSE loss is not below zero, that would be impossible anyway. Additionally, the graph you provided does not match the numbers shown above. Does it come from different runs? Finally, your network is converging to some point, although possibly not ideal. Some suggestions on improving performance:

  1. Your model is quite heavily regularized through Dropout layers, even though it seems you are underfitting (not overfitting). I would suggest removing Dropouts and only adding them later, in case you notice overfitting.
  2. Try training for more epochs, until you actually see performance stabilizing.
  3. Try experimenting with number of LSTM units and LSTM layer stacking and evaluate performance on test set. This has to be an incremental process.
  4. Normalization is part of the experiment. You have to evaluate different techniques to discover which one works better for your task.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.