0
$\begingroup$

I'm new to time-series-forecasting and was wondering, whether in a single variable forecast e.g.: X -> Y the creation of additional features of X leads to an improvement when training. So if adding features like month, day-of-week, rolling mean or lagged series from X does improve Statistical models (Arima, Exponential Smoothing), Tree based models (XGB, Random Forest) or neural networks. These features would be fully correlated with X, so does it help, make predictions worse or have no effect? Thank you in advance

$\endgroup$

2 Answers 2

1
$\begingroup$

Most tree-based models are not a good fit for time series forecasting. I'm not aware of any truly tree-based models that extrapolate outside of the training data as is required for forecasting.

As for the other models you mention, better input features for the particular application will result in better model performance. What makes a "better" features is extremely application dependent. I have used rolling moments with good success in a forecasting application.

Typically ARIMA and exponential smoothing models are univariate. In fact, every state space model (which exponential smoothing models fall under) has an ARIMA equivalent. The same can be said of their multivariate extensions (see Forecasting with Exponential Smoothing: the State Space Approach by Rob Hyndman). I bring up the fact they are univariate because your question seems like you want to feed multiple features to your model.

Features built from a time series do not need to be "fully correlated" with the time series as you say. As an example a transformation from time to something cyclic in time like day of the week or month will not be "fully correlated" with the original input.

$\endgroup$
4
  • $\begingroup$ Tree-based models can be used for forecasting if you reformat the data by windowing, just as you would for neural networks, and can be very powerful for forecasting. $\endgroup$
    – m13op22
    Feb 21 at 21:32
  • $\begingroup$ I see your point. However, I'm still not aware of any tree based models (except for model trees) that are capable of extrapolating outside the range of response variables seen in the training data. A traditional forecasting problem like "given a history of car sales by month, predict car sales for each of the next 12 months" would usually desire that behavior, especially when there is a positive trend in sales. ARIMA and exponential smoothing models can both handle that case, but I'd expect the tree models to predict the maximum value seen in the training set for the next 12 months. $\endgroup$
    – noNameTed
    Feb 22 at 15:05
  • $\begingroup$ @noNameTed GBDTs and random forest are excellent algorithms for modelling time series data. They do well on competitions, and in production for time series forecasting, especially when you have a lot of exogenous variables. They don't need to be in time, you just need to make time into features. $\endgroup$
    – Comte
    Apr 12 at 11:51
  • $\begingroup$ Detrending, helps trees to extrapolate. And if one really wants they can use linear trees (results may vary). $\endgroup$
    – Comte
    Apr 12 at 12:45
1
$\begingroup$

The answer is, it depends on your data.

Converting a univariate forecasting problem to a univariate with exogenous variables can help improve forecasts, particularly for GBDTs. For models that take exogenous variables performance is often (although not always) better with additional features. If features (X) help explain y, they will a model disentangle the patterns in the data. For example, here you see nbeatsx performing better than nbeats. You also mention date features, lags and MAs but you could also use cos and sin waves (fourier terms), these are extremely useful too.

While it is often preferable for exogenous variables to not be correlated with the dependents error terms, this is not a problem for neural networks or DTs which essentially reduce the influence of features that are not helpful in modelling. Worth noting correlated features may be more of an issue for random forest, and that it will affect interpretation even for GBDTs but not prediction. Its also important to point out that time series data is typically auto correlated so you have to draw this out in some way.

For ARIMA etc inherently create lags and moving averages, AR (auto regressive/lags) I (integrated/differencing), MA (moving average), this is also seen in seasonal components when given to a model (SARIMA). So you wouldn't create these directly, but other features may help a SARIMAX (x = exogenous). For example, if we predicted sales, price would help, as would say a promotional condition.

Now if you have noisy data, you may find the additional exogenous do not help, as I stated the tree based models and NNs typically won't be affected so much, so they will largely just create the same solution if the exogenous are included or not. Another solution in such a case is to aggregate your data to a lower temporal resolution.

Hope that helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.