I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use.
There's just one feature Transaction Amount and index of the dataframe is datetime. Data spans past ten years. Around 3500 samples. You can see the data here:
I made the data stationary using diff, dropped the NaN and then I created additional features this way:
def create_features(df):
df['date'] = df.index
df['hour'] = df['date'].dt.hour
df['dayofweek'] = df['date'].dt.dayofweek
df['quarter'] = df['date'].dt.quarter
df['month'] = df['date'].dt.month
df['year'] = df['date'].dt.year
df['dayofyear'] = df['date'].dt.dayofyear
df['dayofmonth'] = df['date'].dt.day
df['weekofyear'] = df['date'].dt.isocalendar().week
X = df[['hour','dayofweek','quarter','month','year',
'dayofyear','dayofmonth','weekofyear']]
y = df["TRANSACTIONAMOUNT"]
return X,y
I used this to separate test and train datasets' y component:
X_train, y_train = create_features(train)
X_test, y_test = create_features(test)
Then I used XGBOOST with different parameters as well as MapieTimeSeriesRegressor.
reg = xgb.XGBRegressor(n_estimators=1000,early_stopping_rounds=50)
reg.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
verbose=True)
produced (after converting back to original using cumsum):
With XGBOOST having hyperparameters tuned:
best_params = {'subsample': 0.6, 'reg_lambda': 0.05, 'reg_alpha': 20, 'n_estimators': 1000, 'min_child_weight': 5, 'max_depth': 10, 'learning_rate': 0.15, 'colsample_bytree': 1.0, 'colsample_bynode': 0.7, 'colsample_bylevel': 0.9}
reg = xgb.XGBRegressor(early_stopping_rounds=50,**best_params)
reg.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
verbose=True)
produced:
and finally, Mapie with and without update of residuals (taken from their tutorial on website):
You can see at least three cycles in every prediction set. These are basically copies of the training set repeated multiple times. How do I get rid of them and make a better prediction?
