Features and LSTM

Question

I have a problem while developing an LStm model. I have 4 feaures that I want to use to make a prediction. When I test my model with a single feaure I get average results but when I test with all 4 feaures the results are even worse. I don't understand where the problem could come from. When we add feaures to the LSTM model the performances should be better. This means that the model is more wrong when it has more variables. I want to clarify that when I test the feaures independently I always have the same performance. My model is really basic.

model = Sequential() 
model.add(LSTM(nb_units,
               input_shape = my_shape
              )) 
Dense(1,activation='sigmoid')  
model.compile(loss='mean_absolute_error',optimizer='adam') 

history = model.fit(X_train, y_train, epochs=ep, validation_data=(X_val, y_val), verbose=2, shuffle=False)

Example results:

• MAE feature 1 : 4
• MAE feature 2 : 4
• MAE feature 3 : 4
• MAE feature 4 : 4
• MAE feature 1,2,3,4 : 7

Do you have any idea where the problem could come from?

Example of my dataset:

So my input data with the feature "pressure" will be ( with the use of a mask):

[ [[12],[0][0]],
  [12],[15][0]],
  [12],[15][12]],
  [[17],[0][0]],
  [17],[17][0]], 
  [17],[17][17]]] ]

Answer 1

Did you normalize your data with a min-max scaler?

LSTM is a complex neuron, and its size should be adapted enough to your data: very simple models could under-perform because LSTMs are not suited for a too limited frame.

To perform well, you should have around 40 neurons and at least 50 timesteps in input.

If your data record is not regular, it is important to transform it into regular intervals, in order to be comparable with other cases.

Therefore, you can apply interpolation to fill the blanks and then take values at regular intervals (for instance every hour). LSTM only works with time data that contains a certain logic in a sequential way. If the records are not regular, LSTM is not able to find the logic of the empty data.

I recommend having a look at the LSTM publication, it has a lot of interesting functions such as Kalman filters, constant error carrousel, etc.

https://www.researchgate.net/publication/13853244_Long_Short-term_Memory