I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use. There's just one feature Transaction Amount and index of the dataframe is datetime. Data spans past ten years. Around 3500 samples. You can see the data here: Test and Training Data

I made the data stationary using diff, dropped the NaN and then I created additional features this way:

def create_features(df):
    df['date'] = df.index
    df['hour'] = df['date'].dt.hour
    df['dayofweek'] = df['date'].dt.dayofweek
    df['quarter'] = df['date'].dt.quarter
    df['month'] = df['date'].dt.month
    df['year'] = df['date'].dt.year
    df['dayofyear'] = df['date'].dt.dayofyear
    df['dayofmonth'] = df['date'].dt.day
    df['weekofyear'] = df['date'].dt.isocalendar().week
    X = df[['hour','dayofweek','quarter','month','year',
    return X,y

I used this to separate test and train datasets' y component:

X_train, y_train = create_features(train)
X_test, y_test = create_features(test)

Then I used XGBOOST with different parameters as well as MapieTimeSeriesRegressor.

reg = xgb.XGBRegressor(n_estimators=1000,early_stopping_rounds=50)
reg.fit(X_train, y_train,
        eval_set=[(X_train, y_train), (X_test, y_test)],

produced (after converting back to original using cumsum): XGBOOSt-simple With XGBOOST having hyperparameters tuned:

best_params = {'subsample': 0.6, 'reg_lambda': 0.05, 'reg_alpha': 20, 'n_estimators': 1000, 'min_child_weight': 5, 'max_depth': 10, 'learning_rate': 0.15, 'colsample_bytree': 1.0, 'colsample_bynode': 0.7, 'colsample_bylevel': 0.9}
reg = xgb.XGBRegressor(early_stopping_rounds=50,**best_params)
reg.fit(X_train, y_train,
        eval_set=[(X_train, y_train), (X_test, y_test)],

produced: XGBOOST-tuned and finally, Mapie with and without update of residuals (taken from their tutorial on website): Mapie You can see at least three cycles in every prediction set. These are basically copies of the training set repeated multiple times. How do I get rid of them and make a better prediction?


1 Answer 1


This is a random walk. Machine learning cannot be used to predict it.

  • $\begingroup$ Why do you think so? $\endgroup$
    – Ben Reiniger
    Commented Oct 22, 2023 at 1:23
  • $\begingroup$ Because it is non-stationary and the first difference produced an autocorrelation with only one peak (the first one). Basically white noise. $\endgroup$ Commented Oct 22, 2023 at 15:45
  • $\begingroup$ Also kudos to the person who downvoted. Nobody helped me for two months and when I posted my findings they got downvoted. $\endgroup$ Commented Oct 22, 2023 at 15:46
  • $\begingroup$ I downvoted the answer, because it gives no reasoning. If you elaborate, I'd be happy to change that. And I'm sorry nobody else answered; I'm not particularly expert on time series, and hadn't seen the question until your answer populated in a review queue. $\endgroup$
    – Ben Reiniger
    Commented Oct 22, 2023 at 17:13
  • $\begingroup$ I am under no obligation to answer time-wasters. Have elaborated, go look it up. $\endgroup$ Commented Oct 22, 2023 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.