0
$\begingroup$

I have been getting poor results on my time series predictions with a LSTM network. I'm looking for any ideas to improve the model.

Predictions VS True Data

The above graph shows the True Data vs. Predictions. The True Data is smooth zig zag shaped, from 0 to 1. However the predictions rarely reach 0 or 1.

The distribution in the prediction data-set rarely reaches 0 or 1 and it's centered around 0.5.

enter image description here

However the distributions in the True Data set is evenly distributed.

enter image description here

Here is the LSTM model built in keras:

model = Sequential()
model.add(Dropout(0.4, input_shape=(train_input_data_NN.shape[1], train_input_data_NN.shape[2])))
model.add(Bidirectional(LSTM(30, dropout=0.4, return_sequences=False, recurrent_dropout=0.4), input_shape=(train_input_data_NN.shape[1], train_input_data_NN.shape[2])))
model.add(Dense(1))
model.compile(loss='mae', optimizer='adam')

How do I get the predictions to be more similar to the true data?

$\endgroup$
  • $\begingroup$ Are you sure there is a lot of signal in there? Can you overfit on your training data? You are using a lot of dropout, maybe try reducing that. $\endgroup$ – Jan van der Vegt Mar 20 '18 at 10:36
  • 1
    $\begingroup$ It is very noisy data, and it does over-fit easily even with this level of dropout. $\endgroup$ – Aaron Isaac Mar 20 '18 at 11:22
  • $\begingroup$ Why aren't you returning sequences, and why are you using bidirectional? If you are predicting at each timestep and you have access to all the features at the whole timeline (for bidirectional) then that would make sense to me, but maybe I'm missing something $\endgroup$ – Jan van der Vegt Mar 20 '18 at 11:40
1
$\begingroup$

I have just used LSTM to train a model predict time series value and get good result as below:

# reshape input to be [samples, time steps, features]
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

model = Sequential()
model.add(LSTM(40, input_shape=(trainX.shape[1], trainX.shape[2])))
model.add(Dense(1))
model.compile(loss='mean_absolute_error', optimizer='adam')
model.fit(trainX, trainY, epochs=10, batch_size=1, verbose=0)

Here is my result: blue is real value, orange is predict.

enter image description here

| improve this answer | |
$\endgroup$
-1
$\begingroup$

Ok it seems i calculated the outputs wrongly. It was not calculated fairly across the entire data-set.

I am getting better results after improving the output calculations:

enter image description here

It still can be improved, but it's a great start.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Can you add some details as to what you did? $\endgroup$ – ubuntu_noob Feb 18 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.