# Time series prediction

I am trying to predict a time serie from another one. My approach is based on a moving windows. I predict the output value of the serie from the following features: the previous value and the 6 past values of the source serie.

Is it usefull to add the previous value of the time serie ?

I feel like I don't use all the information contained in the curve to predict futures values. But I don't see how it would be possible to use all previous data to predict a value (first, the number of features would be growing trough time...).

What are the caveats of a 6 month time-window approach ?

Is there any paper about differents method of feature selection for time-series ?

• When you say you're using a moving window, do you mean you are performing some manipulation of the data within that window or just using the previous 6 values? – Robert Smith Dec 28 '14 at 20:15
• Yes exactly, and the previous value of the serie I am trying to predict. – lcrmorin Dec 28 '14 at 22:54

## 2 Answers

Let me give you a few simple approaches in time series analysis.

The first approach consists in using previous values of your time series $Y_{t}$ as in $Y_{t} = \phi_{1}Y_{t-1} + ... + \phi_{n}Y_{t-n}$. In case you don't know, these models are called autoregressive (AR) models. This answers your first question. Of course it is useful to include the previous value of your time series. There is a whole set of models based on that idea.

The second approach is taking a window and extracting some features to describe the time series at each point in time. Then you use a conventional machine learning technique to predict future values as typically done. This is more common in a classification or regression setting but future values can be thought of as classifying future values. This technique has the advantage of dramatically reducing the number of features, although you usually lose characteristics associated with time. This addresses your second concern.

Another model that could be helpful in your case is the vector autoregressive model (VAR) (using Wikipedia's notation):

$$\left( \begin{array}{ccc} y_{1,t} \\ y_{2,t} \end{array}\right) = \left( \begin{array}{ccc}c_{1} \\ c_{2}\end{array}\right) + \left( \begin{array}{ccc}A_{1,1} & A_{1,2} \\ A_{2,1} & A_{2,2}\end{array}\right)\left( \begin{array}{ccc} y_{1,t-1} \\ y_{2,t-1} \end{array}\right) + \left( \begin{array}{ccc} e_{1,t} \\ e_{2,t} \end{array}\right)$$

Here you can see that $y_{1,t}$ has a contribution from its previous value $t_{1,t-1}$ but also includes the value of the other series $y_{2,t-1}$ in a linear combination. As usual, the purpose is to find the elements of $A_{i,j}$ that minimize some measure of error between observed values and estimated values.

A general suggestion: The first thing you need to do is to test the autocorrelation of your first series in order to confirm that an autoregressive approach is suitable and then test the cross correlation between both series to support the idea that using the second series to improve your predictions is appropriate.

• I strongly suspect non-linear dependencies. That is why I wanted to try Machine Learning on multiple features, their respectives powers and cross products. Your answer is still very helpfull as I just understood that the machine learning process I wanted to do here is strictly the same as fitting a model to my time-serie. (wich I am very familiar with to say the least) – lcrmorin Dec 29 '14 at 14:25
• Sure. If there is a non-linear dependency between series, then a feature based approach is a good approach. – Robert Smith Dec 29 '14 at 20:12

I'm fairly new to this myself, but have spent a lot of time recently learning about time series and hope that I can help fellow learners. If I had the reputation to comment I'd ask you a few things first, but I can't. I'll happily do further work and edit this response if you respond or make edits to your question. With those caveats out of the way:

# Is it useful to use the previous value as a feature?

One of the first things I would say is that the correct aspects to be looking at in your data very much depends on the nature of the data, as well as what you're trying to do with it:

• It sounds like you have monthly values, but it's not clear how far into the future you're wanting to predict, or how much historic data you have access to.
• We also don't know what these two series represent, or why one time series is being used to predict the other - and without that, I don't think anyone will be able to tell you whether the previous value of the series to be predicted is valuable information or not.

# Any caveats to using a 6 month time window?

One obvious caveat to only using the last 6 months is that if there's any seasonality over the year-long period then you're going to miss it.

• If you're not sure: if you have multiple years of information, try plotting the series you want to predict over multiple years. You may well be able to see whether the series generally increases or decreases at certain times of year. If you can share this plot here, it might help people answer your questions in more depth.

As far as caveats about this time-window approach, I'm not too clear from your post what algorithm you're using to predict values. More information on that would be helpful; it's possible that rather than questioning what features to select, you should be questioning what methodology to use for forecasting.

# Helpful further reading?

Once you've provided more information I'll be happy to tackle your last question on suitable reading if I'm able to. For now, I will say that there's a lot of information available, but a lot of it is academic in nature. Quite frequently papers aren't easy to digest, seem to contradict one another, or are only relevant to specific situations. This is a rapidly growing and changing field, so it's sometimes difficult to find a clear best practice or consensus opinion.

That said, it might be worth looking at some of the free, online courses available to see if any would help you understand the area you're interested in:

• The question about previous value was overall usefullness. If the previous value is a function of past rates, how is this different from just taking a bigger window. – lcrmorin Dec 28 '14 at 22:57
• For the algorithm: regularized polynomial regression (Followed Andrew Ng course on Coursera) – lcrmorin Dec 28 '14 at 23:03
• For the time-serie parts: I (more or less) know how to pre-process data. I was wondering if there is algorithm for time series and not a specific number of features. – lcrmorin Dec 28 '14 at 23:06
• Great, I can comment here! Apologies for not being able to ask you questions in the comments before. The time series that you're predicting - does it have any historic data, or is it made up purely of your predictions? – Jo Douglass Dec 28 '14 at 23:23
• @Imorin - I would agree that if you're only looking at the 6 previous values in a time series, you're possibly not using all available information. Have you had a look at Rob Hyndman's blog on forecasting? There's a lot about forecasting and time series, and there's a post from Oot 4 2010 where he discusses cross validation as a method to check on the usefulness of variables in a forecasting model. Time series are mentioned at the end. There are also often helpful discussions in the comments. robjhyndman.com/hyndsight/forecasting – Jo Douglass Dec 29 '14 at 0:30