4
$\begingroup$

I'm using keras for multiple-step ahead time series forecasting of a univariate time series of type float. Judging by the results I got, the approach works works perfectly well. There is, however, an important detail in the training process that baffles me:

keras requires the sequence length of the input sequences (X matrix) to be equal to the forecasting horizon (y matrix). That means, for example, that keras needs input sequences of length 20 in order to forecast the next 20 time steps. My goal is to be able to forecast as many time steps as I specify, given the last 20 time steps. With the below code, this is not possible:

    import numpy as np
    from keras.layers.core import Dense, Activation, Dropout, TimeDistributedDense
    from keras.layers.recurrent import LSTM
    from keras.models import Sequential
    from keras.optimizers import RMSprop
    np.random.seed(1234)


    model = Sequential()
    layers = [1, 20, 40, 1]

    model.add(LSTM(
        input_dim=layers[0],
        output_dim=layers[1],
        return_sequences=True))
    model.add(Dropout(0.3))

    model.add(LSTM(
        layers[2],
        return_sequences=True))
    model.add(Dropout(0.3))

    model.add(TimeDistributedDense(
        output_dim=layers[3]))
    model.add(Activation("linear"))

    rms = RMSprop(lr=0.001)
    model.compile(loss="mse", optimizer=rms)
    model.fit(
        X_train, y_train,
        batch_size=512, nb_epoch=100, validation_split=0.1)

I get a dimension error whenever the sequence lengths of the sequences in X_train and in y_train differ from one another.

What am I doing wrong and how can I fix it? Why are my results still pretty good?

The question is related to this.

$\endgroup$
1
  • $\begingroup$ Have you solved this issue? I am currently trying to do the same thing but cannot seem to get it right. Sebastian $\endgroup$
    – Sebastian
    Jan 13, 2017 at 13:57

1 Answer 1

3
$\begingroup$

No, it is not possible with Vanilla LSTM. It's not a Kera problem, but the fundamental structure of Vanilla LSTM.

I think you have a bit of misunderstanding of how LSTM works. You are assuming given $x_0$...$ x_{19}$ the LSTM is predicting $x_{20}$...$x_{39}$. This is NOT the case. In LSTM if your input sequence is $x_0$...$ x_{19}$ then your output sequence is $x_1$...$ x_{20}$.

To achieve what you want, you can use something like Seq2Seq model.

$\endgroup$
1
  • $\begingroup$ You could feed your predictions back into the network as if they were the observed values, but likely you would end up with very bad estimates after compounding errors many times in a row. $\endgroup$
    – kbrose
    Feb 16, 2018 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.