1
$\begingroup$

I am trying to forecast some sales data with monthly values, I have been trying some classical models as well ML models like XGBOOST.

My data with a feature set looks like this with a length of 110 months and I am trying to forecast for next 12 months,

enter image description here

When it comes to XGBOOST, I've been spending time mostly on hyperparameter optimization with Gridsearch and also state-of-art packages like optuna. My currently best set of parameters looks like this,

parameters = {
            'n_estimators': [700, 1000, 1400],
            'colsample_bytree': [0.7, 0.8],
            'max_depth': [15,20,25],
            'reg_alpha': [1.1, 1.2, 1.3],
            'reg_lambda': [1.1, 1.2, 1.3],
            'subsample': [0.7, 0.8, 0.9],
            'learning_rate': [0.2, 0.3, 0.4],
            'min_child_weight': [1]}
        
        skrg = XGBRegressor(objective = 'reg:linear')
        
        fit_params={"early_stopping_rounds":50, 
            "eval_metric" : "rmse", 
            "eval_set" : [[X_test, y_test]]}
        
        search_sk = GridSearchCV (
        skrg, parameters, cv=TimeSeriesSplit(n_splits=cv).get_n_splits([X_train, y_train]),
                        scoring='neg_mean_squared_error',
                            ) # 5 fold cross validation
        search_sk.fit(X_train, y_train, **fit_params)

with results like, enter image description here enter image description here

I couldn't figure out how to model for the upwards trend in the data. Does it come under the optimization or do I have to do something with my data like transformations or adding extra features. Any advices? Thanks!

$\endgroup$

2 Answers 2

1
$\begingroup$

Before fit XGBOOST you should make timeseries stationary, here you can find more info about that.

Or you can try linear models, like Linear or Logistic Regression, they are find trends much better.

$\endgroup$
1
  • $\begingroup$ Thank you, I however do not understand how stationrizing the data will help with trend, As I understand, it only reduces variance in the data, However I will try some transformations like box-cox and come back with how the results look. $\endgroup$ Jun 3, 2022 at 7:17
1
$\begingroup$

Linear regression excels at extrapolating trends, but can't learn interactions. XGBoost excels at learning interactions, but can't extrapolate trends.

You can create a "hybrid" forecasters that combine complementary learning algorithms and let the strengths of one make up for the weakness of the other.

Source: https://www.kaggle.com/code/ryanholbrook/hybrid-models

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.