Time series prediction

Question

I am trying to predict a time serie from another one. My approach is based on a moving windows. I predict the output value of the serie from the following features: the previous value and the 6 past values of the source serie.

Is it usefull to add the previous value of the time serie ?

I feel like I don't use all the information contained in the curve to predict futures values. But I don't see how it would be possible to use all previous data to predict a value (first, the number of features would be growing trough time...).

What are the caveats of a 6 month time-window approach ?

Is there any paper about differents method of feature selection for time-series ?

Answer 1

Let me give you a few simple approaches in time series analysis.

The first approach consists in using previous values of your time series $Y_{t}$ as in $Y_{t} = \phi_{1}Y_{t-1} + ... + \phi_{n}Y_{t-n}$. In case you don't know, these models are called autoregressive (AR) models. This answers your first question. Of course it is useful to include the previous value of your time series. There is a whole set of models based on that idea.

The second approach is taking a window and extracting some features to describe the time series at each point in time. Then you use a conventional machine learning technique to predict future values as typically done. This is more common in a classification or regression setting but future values can be thought of as classifying future values. This technique has the advantage of dramatically reducing the number of features, although you usually lose characteristics associated with time. This addresses your second concern.

Another model that could be helpful in your case is the vector autoregressive model (VAR) (using Wikipedia's notation):

$$\left( \begin{array}{ccc} y_{1,t} \\ y_{2,t} \end{array}\right) = \left( \begin{array}{ccc}c_{1} \\ c_{2}\end{array}\right) + \left( \begin{array}{ccc}A_{1,1} & A_{1,2} \\ A_{2,1} & A_{2,2}\end{array}\right)\left( \begin{array}{ccc} y_{1,t-1} \\ y_{2,t-1} \end{array}\right) + \left( \begin{array}{ccc} e_{1,t} \\ e_{2,t} \end{array}\right)$$

Here you can see that $y_{1,t}$ has a contribution from its previous value $t_{1,t-1}$ but also includes the value of the other series $y_{2,t-1}$ in a linear combination. As usual, the purpose is to find the elements of $A_{i,j}$ that minimize some measure of error between observed values and estimated values.

A general suggestion: The first thing you need to do is to test the autocorrelation of your first series in order to confirm that an autoregressive approach is suitable and then test the cross correlation between both series to support the idea that using the second series to improve your predictions is appropriate.

Answer 2

I'm fairly new to this myself, but have spent a lot of time recently learning about time series and hope that I can help fellow learners. If I had the reputation to comment I'd ask you a few things first, but I can't. I'll happily do further work and edit this response if you respond or make edits to your question. With those caveats out of the way:

Is it useful to use the previous value as a feature?

One of the first things I would say is that the correct aspects to be looking at in your data very much depends on the nature of the data, as well as what you're trying to do with it:

• It sounds like you have monthly values, but it's not clear how far into the future you're wanting to predict, or how much historic data you have access to.
• We also don't know what these two series represent, or why one time series is being used to predict the other - and without that, I don't think anyone will be able to tell you whether the previous value of the series to be predicted is valuable information or not.

Any caveats to using a 6 month time window?

One obvious caveat to only using the last 6 months is that if there's any seasonality over the year-long period then you're going to miss it.

• If you're not sure: if you have multiple years of information, try plotting the series you want to predict over multiple years. You may well be able to see whether the series generally increases or decreases at certain times of year. If you can share this plot here, it might help people answer your questions in more depth.

As far as caveats about this time-window approach, I'm not too clear from your post what algorithm you're using to predict values. More information on that would be helpful; it's possible that rather than questioning what features to select, you should be questioning what methodology to use for forecasting.

Helpful further reading?

Once you've provided more information I'll be happy to tackle your last question on suitable reading if I'm able to. For now, I will say that there's a lot of information available, but a lot of it is academic in nature. Quite frequently papers aren't easy to digest, seem to contradict one another, or are only relevant to specific situations. This is a rapidly growing and changing field, so it's sometimes difficult to find a clear best practice or consensus opinion.

That said, it might be worth looking at some of the free, online courses available to see if any would help you understand the area you're interested in:

• Coursera "data science" related courses