2
$\begingroup$

Theoretical Noob question -

Can we use boosting methods to effectively forecast the future after being trained on a non-stationary time series? Or do you train/fit on the residual of the training set and then add seasonality/trend components while forecasting?

Thanks in advance.

$\endgroup$
1
  • 1
    $\begingroup$ Tree-based models don't follow any parameter-based relation of Features e.g. LinearRegression Or NeuralNetwork. It simply divides the training data into finer spaces, to have a minimum possible impurity. So, it will definitely miss the Trend. $\endgroup$ – 10xAI Jan 25 at 16:11
1
$\begingroup$

As @10xAI said, a tree-based gradient boosted approach may miss the mark for time series because it cannot forecast a growing trend. However, we can apply gradient boosting methodology to any algorithm. You can mess around with some code I wrote that is based on gradient boosting and decomposition: LazyProphet. The code is badly written and I think the example data pulls break now but the method itself tends to produce some decent results.

Essentially if we do boosting with some piecewise approach we can get new changepoints at each boosting round and update our seasonality + exogenous measures. I use binary segmentation so it ends up being very similar to a wild binary segmentation approach for change points. Round 0 the 'trend' is just the mean/median, then you measure seasonality and set your 'y' to the original time series - (trend + seasonality). Round 1 then finds the optimal point which splits the data into 2 and fits a trend estimator (could be the mean kind of like a tree output) then measures seasonality and adds these measures to what was found in round 0 to find the new residuals to fit for the next round and so on. Hopefully this image makes it clearer: enter image description here

I do have a much better written and more generalized approach coming soon!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.