XGBoost training on sample of time series data

Question

I am new to XGBoost and would like to use it on a time series dataset. Here is the scenario I'm faced with:

The data set contains N samples of length T, with N>>T. I'd like to train an XGBoost model to predict the next H values, with H<<T. So the input has shape (N,T) and the labels has shape (N,H). However, it appears that XGBRegressor (see docs) requires labels to be shape (N,).

My question: How would one train using all N samples to predict the next H values?

Answer 1

More generally seen, your problem is, that you want to predict a multi-variate / multi-dimensioan lable, but your algorithm only supports uni-variate / 1-dimensional target variables.

I see two direct approaches that you could try:

Independent Models

You could train $H$ independent XGBoost Models, one for each target dimension. sklearn already provides a wrapper for this:

model = MultiOutputRegressor(XGBRegressor())
model.fit(X, y)

Flatten the output

You could turn each sample into $H$ different samples, one for each output dimension. In order to distinguish these dimensions, you could add the index of the output as another feature. In other word, your $(N,T)$ input would be transformed into an $(N\cdot H, T+1)$ input and your target into an $(N\cdot H)$ vector.

For example, the data

X = [[1, 5, -3], 
     [2, 4, 6]]
y = [[4, 6], 
     [13, 20]] 

would be transformed into

X = [[1, 5, -3, 1], 
     [1, 5, -3, 2], 
     [2, 4, 6, 1],
     [2, 4, 6, 2]]
y = [4, 6,13,20]