How to make XGBOOST capture trend in time series forecasting?

Question

I am trying to forecast some sales data with monthly values, I have been trying some classical models as well ML models like XGBOOST.

My data with a feature set looks like this with a length of 110 months and I am trying to forecast for next 12 months,

When it comes to XGBOOST, I've been spending time mostly on hyperparameter optimization with Gridsearch and also state-of-art packages like optuna. My currently best set of parameters looks like this,

parameters = {
            'n_estimators': [700, 1000, 1400],
            'colsample_bytree': [0.7, 0.8],
            'max_depth': [15,20,25],
            'reg_alpha': [1.1, 1.2, 1.3],
            'reg_lambda': [1.1, 1.2, 1.3],
            'subsample': [0.7, 0.8, 0.9],
            'learning_rate': [0.2, 0.3, 0.4],
            'min_child_weight': [1]}
        
        skrg = XGBRegressor(objective = 'reg:linear')
        
        fit_params={"early_stopping_rounds":50, 
            "eval_metric" : "rmse", 
            "eval_set" : [[X_test, y_test]]}
        
        search_sk = GridSearchCV (
        skrg, parameters, cv=TimeSeriesSplit(n_splits=cv).get_n_splits([X_train, y_train]),
                        scoring='neg_mean_squared_error',
                            ) # 5 fold cross validation
        search_sk.fit(X_train, y_train, **fit_params)

with results like,

I couldn't figure out how to model for the upwards trend in the data. Does it come under the optimization or do I have to do something with my data like transformations or adding extra features. Any advices? Thanks!

Answer 1

Before fit XGBOOST you should make timeseries stationary, here you can find more info about that.

Or you can try linear models, like Linear or Logistic Regression, they are find trends much better.

Answer 2

Linear regression excels at extrapolating trends, but can't learn interactions. XGBoost excels at learning interactions, but can't extrapolate trends.

You can create a "hybrid" forecasters that combine complementary learning algorithms and let the strengths of one make up for the weakness of the other.

Source: https://www.kaggle.com/code/ryanholbrook/hybrid-models