3
$\begingroup$

I was wondering if anyone has experience with time series prediction for data from multiple sources. So for instance, time series $a,b,..,z$ each have their own shape, some may be correlated with others. The ultimate goal is to have a model trained such that the value at time $t+1$ for any given data source can be predicted.

I personally have two solutions that in theory could work, but was wondering if anyone knew of other frequently used methods.

  1. Multi-task learning with LSTM

  2. Use feature engineering to model properties of each time series source as features along with usual features and use these with LSTM

Improve this question
$\endgroup$
2
  • $\begingroup$ What are time series "from multiple sources"? What is the "own shape" of a time series? $\endgroup$
    – tagoma
    Commented Jul 17, 2017 at 11:16
  • $\begingroup$ so for instance, I have timeseries representing sales of multiple groups of supermarket product. A single sequence represents the sales of Product group X over a 5 year window. The sales curve of each individual product group id different, yet, there may be correlations between the sales of multiple groups. I tried lumping all of these together and using a few feature that would capture the statistical characteristics of that feature for both a long and short window, but the results were not great. Running predictions for each individual sequence seems to work fine (as expected) $\endgroup$
    – Nmaple
    Commented Jul 19, 2017 at 15:46

3 Answers 3

Reset to default
1
$\begingroup$

I'm only aware (and using) a RNN which gets multiple time-series in it's first layer and then mixes those in the following layers. Let me know if you have a question on this approach.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you. Could you please read my comment above and see if you were also successful in using multiple time-series with different sources (in my case product types) in the same model? if so, did you have to do any form of feature engineering to take into account the characteristics of each individual sequence or did you apply another method? $\endgroup$
    – Nmaple
    Commented Jul 19, 2017 at 15:48
  • $\begingroup$ I have not done exactly that but there should be no general problem. I wouldn't do feature engineering. In general feature engineering is not needed for deep learning, sure there might be situations when it helps. $\endgroup$
    – Tobi
    Commented Jul 21, 2017 at 11:45
1
$\begingroup$

You can certainly use an LSTM for this approach, as well as VAR, or potentially a more 'traditional' model (random forest). I have implemented multivariate LSTM for t+x prediction in Keras. Feature engineering may definitely help improve results.

I have also used VAR and was surprised at the effectiveness of this when compared against the LSTM. There are packages in Python and R for VAR.

Alternatively, if your features are effectively describing a pattern that is not necessarily dependent on the time component, a more traditional method such as random forest could be used. Probably not as effective as VAR or LSTM, but it's fast.

Improve this answer
$\endgroup$
1
  • $\begingroup$ thanks. were you able to use multi-source data (described in the above comment) in your example? I am able to predict t+1 for a single source timeseries. I am reluctant to call it uni-variate as it has multiple features but corresponds to a specific product group. In my case, the input for a single source prediction to keras looks like (100,5,20) $\endgroup$
    – Nmaple
    Commented Jul 19, 2017 at 15:58
1
$\begingroup$

The answer is Data Fusion or Feature Fusion.

I am implementing one using neural networks to classify Human Activity using multiple kinds of sensors: accelerometer, binary sensors, ecc..

We train a neural network with multiple input layers: one for each data source.

The forward pass keeps these separate until deeper in the network, at which point we concatenate the feature representations and then proceed with a single, merged processing pipeline in the network which ends at a single classification layer.

We can think of it as each independent pipeline learn a feature representation of its data source that can then be usefully combined with the other representations to perform classification.

This maintains independence of each network branch (e.g. you could have an LSTM processing sequential data, whose latent state vector then is concatenated with a FC feature from a CNN processing pressure plate measurements as images, and so on).

This is the structure I am using

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.