7
$\begingroup$

I have continuous (time series) data. This data is multivariate. Each feature can be represented as time series (they are all calculated on a daily basis). Here is an example:

 Days    F1 F2  F3  F4   F5 Target
Day 1    10  1  0.1 100 -10   1
Day 2    20  2  0.2 200 -20   1
Day 3    30  3  0.3 300 -30   0
Day 4    40  4  0.4 400 -40   1
Day 5    50  5  0.5 500 -50   1
Day 6    60  6  0.6 600 -60   1
Day 7    70  7  0.7 700 -70   0
Day 8    80  8  0.8 800 -80   0

F1, F2, .. F5 are my features, and Target is my binary classes. If I use a window size of 3, I can convert my features into time-series data. Then, I will have [10,20,30] for feat_1, [1,2,3] for feat_2 and so on. With the window size of 3, I have 5 features* 3 window_size, a total of 15 features if written in the same vector.

The problem with this method is putting them into the same vector might cause some problems since the feature values are different.

Example of multivariate time series (15 features in 1 network):

[10, 20, 30, 1, 2, 3, 0.1, 0.2, 0.3, 100, 200, 300, -10, -20, -30]
[20, 30, 40, 2, 3, 4, 0.2, 0.3, 0.4, 200, 300, 400, -20, -30, -40]
....
[60, 70, 80, 6, 7, 8, 0.6, 0.7, 0.8, 600, 700, 800, -60, -70, -80]

The other option is to create separate time series networks (RNNs mostly, LSTM or CNN or their combination) for each of the features with the same target and then combine their results. In this scenario, I have 5 different networks, and all of them are univariate time series binary predictions.

Example of different networks with univariate time series data (3 features in 5 networks):

[10, 20, 30]
...                            This is for network 1
[60, 70, 80]

[1, 2, 3]
...                            This is for network 2
[6, 7, 8]

...

[-10, -20, -30]
...                            This is for network 5
[-60, -70, -80] 

The problem with this one is that I might lose the feature correlation information even though I'm putting their results into another network.

My question is, which is the best way to use when dealing with multivariate time series problems? I want to use the first method, but value differences worry me. The second method is easier, but I worry I might lose some essential information.

$\endgroup$

1 Answer 1

3
$\begingroup$

You can add all features as input to RNN/LSTM (Day #, F1, F2, ... F5) and binary class as output.

This article has an example of such network.

$\endgroup$
9
  • $\begingroup$ Thanks. I have his book about time series forecasting and aware of this link. But what about my main problem with this approach? I mean how will the value difference among features affect the system? $\endgroup$
    – iso_9001_
    Feb 16, 2019 at 14:30
  • $\begingroup$ This should not matter since values for the same feature should be in the same scale (E.g.: all values for F3 seem to be between 0 to 1). $\endgroup$ Feb 16, 2019 at 14:34
  • $\begingroup$ Also, most libraries have option of normalizing values. You can try training with and without normalizing so that difference in scale of F3 and F4 does not slow down training. $\endgroup$ Feb 16, 2019 at 14:36
  • $\begingroup$ OK, got it. I should give it a try. What about method 2? Isn't it better to separate features so that valur differences won't have an effect on the system? $\endgroup$
    – iso_9001_
    Feb 16, 2019 at 14:59
  • $\begingroup$ Recommendation is not not use either Method 1 or 2. Just provide all features and (and last N samples in sequence) to model. With method 2, model will not be able to learn from features such as F2/F5 or F2^2/F1+F5 $\endgroup$ Feb 16, 2019 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.