1
$\begingroup$

I'm trying to make forecasts using sktime for my entire training data and an arbitrary length of out-of-sample data but can't figure it out.

# Generate 2 years of daily data
data = np.random.random(365 * 2,)
df = pd.DataFrame({'y': data})
# Arbirtrary X variable (8% per year growth as a daily increase)
df['daily_growth'] = 8 / 365 

# Forecast for entire dataset and 1 year into the future
fh = np.arange(-len(df)+1, 365+1)
# Fit model
arima = AutoARIMA()
arima.fit(df.y, X=df.daily_growth)
# Create forecast df for in-sample and out-of-sample data... 
# ...this is probably where the problem lies
forecast_df = pd.DataFrame(index=range(len(fh))) # `index=fh` also fails
forecast_df['daily_growth'] = 8 / 365
# ValueError...
preds_with_X = arima.predict(fh=fh, X=forecast_df)

Output

ValueError                                Traceback (most recent call last)
Input In [3], in <cell line: 15>()
     13 preds_no_X = arima_no_X.predict(fh=fh)
     14 len(fh) == len(forecast_df)
---> 15 preds_with_X = arima_with_X.predict(fh=fh, X=forecast_df)
     17 plt.plot(df.y, label='Actual')
     18 plt.plot(preds_no_X, label='preds_no_X')

File ~/opt/anaconda3/envs/humbl_keywords/lib/python3.9/site-packages/sktime/forecasting/base/_base.py:318, in BaseForecaster.predict(self, fh, X)
    316 # we call the ordinary _predict if no looping/vectorization needed
    317 if not self._is_vectorized:
--> 318     y_pred = self._predict(fh=fh, X=X_inner)
    319 else:
    320     # otherwise we call the vectorized version of predict
    321     y_pred = self._vectorize("predict", X=X_inner, fh=fh)

File ~/opt/anaconda3/envs/humbl_keywords/lib/python3.9/site-packages/sktime/forecasting/base/adapters/_pmdarima.py:84, in _PmdArimaAdapter._predict(self, fh, X)
     81 # both in-sample and out-of-sample values
     82 else:
     83     y_ins = self._predict_in_sample(fh_ins, X=X)
---> 84     y_oos = self._predict_fixed_cutoff(fh_oos, X=X)
     85     return pd.concat([y_ins, y_oos])

File ~/opt/anaconda3/envs/humbl_keywords/lib/python3.9/site-packages/sktime/forecasting/base/adapters/_pmdarima.py:177, in _PmdArimaAdapter._predict_fixed_cutoff(self, fh, X, return_pred_int, alpha)
    162 """Make predictions out of sample.
    163 
    164 Parameters
   (...)
    174 Returns series of predicted values.
    175 """
    176 n_periods = int(fh.to_relative(self.cutoff)[-1])
--> 177 result = self._forecaster.predict(
    178     n_periods=n_periods,
    179     X=X,
    180     return_conf_int=False,
    181     alpha=DEFAULT_ALPHA,
    182 )
    184 fh_abs = fh.to_absolute(self.cutoff)
    185 fh_idx = fh.to_indexer(self.cutoff)

File ~/opt/anaconda3/envs/humbl_keywords/lib/python3.9/site-packages/pmdarima/utils/metaestimators.py:53, in _IffHasDelegate.__get__.<locals>.<lambda>(*args, **kwargs)
     50         attrgetter(self.delegate_names[-1])(obj)
     52 # lambda, but not partial, allows help() to work with update_wrapper
---> 53 out = (lambda *args, **kwargs: self.fn(obj, *args, **kwargs))
     54 # update the docstring of the returned function
     55 update_wrapper(out, self.fn)

File ~/opt/anaconda3/envs/humbl_keywords/lib/python3.9/site-packages/pmdarima/arima/auto.py:257, in AutoARIMA.predict(self, n_periods, X, return_conf_int, alpha, **kwargs)
    247 @if_has_delegate("model_")
    248 def predict(self,
    249             n_periods=10,
   (...)
    254 
    255     # Temporary shim until we remove `exogenous` support completely
    256     X, _ = pm_compat.get_X(X, **kwargs)
--> 257     return self.model_.predict(
    258         n_periods=n_periods,
    259         X=X,
    260         return_conf_int=return_conf_int,
    261         alpha=alpha,
    262     )

File ~/opt/anaconda3/envs/humbl_keywords/lib/python3.9/site-packages/pmdarima/arima/arima.py:785, in ARIMA.predict(self, n_periods, X, return_conf_int, alpha, **kwargs)
    783 X = self._check_exog(X)  # type: np.ndarray
    784 if X is not None and X.shape[0] != n_periods:
--> 785     raise ValueError('X array dims (n_rows) != n_periods')
    787 # f = self.arima_res_.forecast(steps=n_periods, exog=X)
    788 arima = self.arima_res_

ValueError: X array dims (n_rows) != n_periods

Alas, pmdarima doesn't print what input it receives for n_rows and n_periods. But I think I am passing the correct shapes.

len(fh) == len(forecast_df)  # True
fh.shape, forecast_df.shape  # ((1095,), (1095, 1))

P.S. I'm not sure my daily_growth var would actually have any impact on the results. Advice on this point and how to get the model to have 8% growth would be helpful too!

$\endgroup$

2 Answers 2

1
$\begingroup$

short answer: here they proposed to use:

This is the correct way to generate predictions by providing the exogenous data:

forecast = model.predict(n_periods=len(df_test), >exogenous=df_test[exogenous_features])

When you are missing the exogenous data, hence the error (X array should contain >your exogenous_features):

model.predict(n_periods=365) 

Long answer:

In my experiment to reach out-of-sample forecasting (predicting beyond the training dataset) over future data (unseen test-set), I separated your generated data into train, test.

There are lots of posts without accepted answers for this topic.

There is no solid guide on how one can use regressors for the concept of out-of-sample forecasting:

Nevertheless, I found some good workarounds like:

  • img

...model generates forecasts for the next 7 days, only the 7-day-ahead forecast makes efficient use of the past data ref

I checked package documentation, but I could not find how one can use out-of-sample using this library as well as documentation.

Note: As it is mentioned here by Jason Brownlee, one of the approaches is Multi-Step Out-of-Sample Forecast which is available in lib under the title of Recursive multi-step forecasting which you can see how:

Time series transformation, including an exogenous variable.

It is unclear to me which parameter (maybe lag or step in ) in those packages we should use with(out) exogenous variable to achieve out-of-sample forecasting i.e. for the next 10 steps.

just note that here there is good workaround about impact of exogenous variable.

Some mainly old ARIMA-related posts help for our goal:

I just found ARIMA-based example in package that corresponds clearly to the parameter of out-of-sample: out_of_sample_size=10 as follows:

#!pip install sktime
#!pip install pmdarima
#!pip install sktime[all_extras]
import warnings
import numpy as np
import pandas as pd
import pmdarima as pm
from pmdarima import model_selection
import matplotlib.pyplot as plt

# hide warnings
warnings.filterwarnings("ignore")

# Generate data
data = np.random.random(365 * 2,)
df   = pd.DataFrame({'y': data})
df['daily_growth'] = np.random.random(365 * 2,)
#fh = [1,2,3,4] #forecast horizon object

#df
###################################
#|    |        y |   daily_growth |
#|---:|---------:|---------------:|
#|  0 | 0.213322 |     0.394769   |
#|  1 | 0.878436 |     0.00674871 |
#|  2 | 0.174274 |     0.759531   |
#|  3 | 0.212685 |     0.23475    |
#|  4 | 0.86434  |     0.0325546  |

# #############################################################################
# Load the data and split it into separate pieces
train, test = model_selection.train_test_split(data, train_size=600)

# #############################################################################
# Fit with some validation (cv) samples
arima = pm.auto_arima(train, start_p=1, start_q=1, d=0, max_p=5, max_q=5,
                      out_of_sample_size=10, suppress_warnings=True,
                      stepwise=True, error_action='ignore')

# Now plot the results and the forecast for the test set
preds, conf_int = arima.predict(n_periods=test.shape[0],
                                return_conf_int=True)

fig, axes = plt.subplots(2, 1, figsize=(12, 8))


x_axis = np.arange(train.shape[0] + preds.shape[0])
axes[0].plot(x_axis[:train.shape[0]], train, alpha=0.75 , label='train')
axes[0].scatter(x_axis[train.shape[0]:], preds, alpha=0.4, marker='o', label='forecast' ,color='m')
axes[0].scatter(x_axis[train.shape[0]:], test, alpha=0.4, marker='x', label='test',color='red')
axes[0].fill_between(x_axis[-preds.shape[0]:], conf_int[:, 0], conf_int[:, 1],
                     alpha=0.2, color='m' ,label = 'interval')

# fill the section where we "held out" samples in our model fit

axes[0].set_title("Train samples & forecasted test samples")

axes[0].legend(loc='upper left')

# Now add the actual samples to the model and create NEW forecasts
arima.update(test)
new_preds, new_conf_int = arima.predict(n_periods=100, return_conf_int=True)
new_x_axis = np.arange(data.shape[0] + 100)

axes[1].plot(new_x_axis[:data.shape[0]], data, alpha=0.75 , label='train')
axes[1].scatter(new_x_axis[data.shape[0]:], new_preds, alpha=0.4, marker='o' , label='new forecast' ,color='lime')
axes[1].fill_between(new_x_axis[-new_preds.shape[0]:],
                     new_conf_int[:, 0],
                     new_conf_int[:, 1],
                     alpha=0.2, color='lime',label = 'interval')
axes[1].set_title("Added new observed values with new forecasts")

axes[1].legend(loc='upper left')
plt.show()

img

$\endgroup$
0
$\begingroup$

You are making two mistakes:

  1. you train a model with a constant exogenous variable.
  2. your forecast horizon object fh does not make sense as it has values <= 0.

This code works:

from sktime.forecasting.arima import AutoARIMA
import pandas as pd
import numpy as np

data = np.random.random(365 * 2,)
df = pd.DataFrame({'y': data})
df['daily_growth'] = np.random.random(365 * 2,)
fh = [1,2,3,4]

arima = AutoARIMA()
arima.fit(df.y, X=df.daily_growth)

forecast_df = pd.DataFrame(index=range(len(fh)))
forecast_df['daily_growth'] = 8 / 365

preds_with_X = arima.predict(fh=fh, X=forecast_df.daily_growth)
$\endgroup$
1
  • $\begingroup$ I get ValueError: X array dims (n_rows) != n_periods. Received n_rows=0 and n_periods=4 $\endgroup$
    – Mario
    Oct 2, 2023 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.