0
$\begingroup$

The time series dataset I am working on has missing samples. I am trying to use keras and LSTM for prediction. How should I handle the missing timestamp samples ( sometimes there are missing weeks even months)?

Does this affect the overall performance of the LSTM predictor?

$\endgroup$
2
$\begingroup$

If you have missing data you can add a Masking layer to your model which will prevent contributions from those time steps to be included. You can see the documentation for it here: https://keras.io/layers/core/#masking

Example taken from the keras documentation:

Consider a Numpy data array x of shape (samples, timesteps, features), to be fed to an LSTM layer. You want to mask timestep #3 and #5 because you lack data for these timesteps. You can:

set x[:, 3, :] = 0. and x[:, 5, :] = 0. insert a Masking layer with mask_value=0. before the LSTM layer:

With example code:

model = Sequential()
model.add(Masking(mask_value=0., input_shape=(timesteps, features)))
model.add(LSTM(32))
$\endgroup$
1
  • 1
    $\begingroup$ Thanks for the answer! I am going to try the masking method. $\endgroup$
    – LexByte
    Jan 25 '19 at 13:29
0
$\begingroup$

Yes it affects the overall performance. As you say it is time series data, I suggest replacing the missing values with geometric mean of the populated values.

Alternatively, you can delete the variables that contain missing values if they are correlated with fully populated variables.

$\endgroup$
1
  • $\begingroup$ Its not just values. There are samples where lets say Jan 20 2017 to April 25 2017 rows are missing. What shall i do about this? $\endgroup$
    – LexByte
    Jan 25 '19 at 12:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.