2
$\begingroup$

I'm struggling to find a tutorial/example which covers using an seq2seq model for sequential inputs other then text/translation.

I have a multivariate dataset with n number of input variables each composed of sequences, and a single output sequence which is unrelated to any of the input variables (e.g. using weekly wind speed and humidity to predict temperature).

I've converted the features and label to batches with fixed time steps and n dimensions, however I'm confused which model should be used. Ideally the output should be a single sequence (e.g. annual temperature) however which model would achieve this? Is this something an LSTM could achieve, or is this a problem for a seq2seq model?

Any suggestions/insightful would be appreciated.

| improve this question | | | | |
$\endgroup$
0
$\begingroup$

The type of your problem is time series regression. The common way of dealing with such problems is using ARMA models and Kalman filters with input (control vector). Look at this links:

https://www.sas.upenn.edu/~fdiebold/Teaching104/Ch14_slides.pdf

https://en.m.wikipedia.org/wiki/Kalman_filter

For time series regression using LSTM, check Multivariate LSTM Forecast Model part from this link:

https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras

| improve this answer | | | | |
$\endgroup$
  • $\begingroup$ What about using a Sequential model in Keras? Would that also be suited to this problem? I've read that ARMA models might need a lot of tuning... $\endgroup$ – Elliot Aug 20 '18 at 15:55
  • $\begingroup$ @Elliot I updated my answer. Please check it. $\endgroup$ – pythinker Aug 20 '18 at 16:44
0
$\begingroup$

To use a seq2seq neural network for timeseries regression (and not forecasting, as everybody seems to be doing), a simple Keras model could be

inputs = Input(shape=(n_timesteps, n_features))
x = LSTM(n_units, return_sequences = True)(inputs)
outputs = Dense(1, activation="linear")(x)

where you output a single value in the final layer, with linear activation, so that no scaling is applied. Then a common choice is to minimize MSE with your favorite optimizer - usually RMSprop for timeseries.

Then, if you add convolutions and pooling layers, be sure to set padding = "same", to maintain the sequence length throughout the layers, or fix downsampled timesteps with RepeatVector to match the input sequence length. Specific choice of layers may be problem-dependent.

| improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.