1
$\begingroup$

I have a problem while developing an LStm model. I have 4 feaures that I want to use to make a prediction. When I test my model with a single feaure I get average results but when I test with all 4 feaures the results are even worse. I don't understand where the problem could come from. When we add feaures to the LSTM model the performances should be better. This means that the model is more wrong when it has more variables. I want to clarify that when I test the feaures independently I always have the same performance. My model is really basic.

model = Sequential() 
model.add(LSTM(nb_units,
               input_shape = my_shape
              )) 
Dense(1,activation='sigmoid')  
model.compile(loss='mean_absolute_error',optimizer='adam') 

history = model.fit(X_train, y_train, epochs=ep, validation_data=(X_val, y_val), verbose=2, shuffle=False)

Example results:

  • MAE feature 1 : 4
  • MAE feature 2 : 4
  • MAE feature 3 : 4
  • MAE feature 4 : 4
  • MAE feature 1,2,3,4 : 7

Do you have any idea where the problem could come from?

Example of my dataset:

So my input data with the feature "pressure" will be ( with the use of a mask):

[ [[12],[0][0]],
  [12],[15][0]],
  [12],[15][12]],
  [[17],[0][0]],
  [17],[17][0]], 
  [17],[17][17]]] ]
$\endgroup$
2
  • $\begingroup$ try using subsets of features eg {1,2}, {3,4}, {2,3} and so on.. Possibly some features are incompatible together so performance is worse $\endgroup$
    – Nikos M.
    Jun 29 at 15:02
  • $\begingroup$ @NikosM. I also tried the combinations but it gives me the same performance as with a single feature $\endgroup$
    – Fancq
    Jun 29 at 15:39

1 Answer 1

1
$\begingroup$

Did you normalize your data with a min-max scaler?

LSTM is a complex neuron, and its size should be adapted enough to your data: very simple models could under-perform because LSTMs are not suited for a too limited frame.

To perform well, you should have around 40 neurons and at least 50 timesteps in input.

If your data record is not regular, it is important to transform it into regular intervals, in order to be comparable with other cases.

Therefore, you can apply interpolation to fill the blanks and then take values at regular intervals (for instance every hour). LSTM only works with time data that contains a certain logic in a sequential way. If the records are not regular, LSTM is not able to find the logic of the empty data.

I recommend having a look at the LSTM publication, it has a lot of interesting functions such as Kalman filters, constant error carrousel, etc.

https://www.researchgate.net/publication/13853244_Long_Short-term_Memory

$\endgroup$
2
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – Stephen Rauch
    Jun 30 at 15:15
  • $\begingroup$ @Nicolas Hello, I am trying the link you sent me. I have two problems that I'm having trouble solving. The first concerns the number of variables, in the tutorial they use only one variable and I don't see how to integrate other variables. Then, following the tutorial I saw that it was necessary with sequences of the same sizes, while in my problem I have sequences of information of increasing sizes because the information is retrieved at different times. $\endgroup$
    – Fancq
    Jul 3 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.