Time-series feature enrichment before or after train-test split?

Question

I am dealing with a time-series that represents the CPU usage registered on Azure Virtual Machine. The historical data include a period of 19 months and its granularity is a 10 minute one (each 10 minute the CPU usage level had been registered). My principal objective is the long-term (one week ahead) forecasting of trend. At the beggining, only one column - usageLevel is available in my raw data set. Naturally, before struggling with any prediction model (I am about to test XGBoost, LSTM, transformers etc.) the common practice is to perform a wide feature enrichment. There are multiple strategies and ideas recommended - some of them include moving average features and calendar ones.

I have decided to use 15 months as training set and the rest (4 months) as testing set. To avoid any leakage of information from training set to the testing one, should I perform feature enrichment (and outliers handling/removal) only after split and only against the training set? My concern is how the models will cope with the different shape of training and test data if I perform data enrichment only against the training set - or just after the split the same set of transformations should be performed against test set to preserve the same shapes? Thanks in advance!

Answer 1

there are misunderstandings of concept here. If you have a complete data set, from 1/1/2021 to 10/31/2022, then the training data will be, for example, from 1/1/2021 to 07/31/2022, use the last part, from 01/08/2022 to 10/31/2022, called "holding sample", to compare the final score between the actual and the predicted value.

Same features (predictors) and different rows (observations) .

So preprocessing and enrichment will be before splitting into training/test, because it ensures that you have the same condition on the data (standardization, removal of leak information, outliers, etc.)

When your model is in production, the data will be involved in the same steps for data pre-processing, called Pipeline, after that, the model called in your web application will be able to make forecasts.

And yes, it's fine to using 15 months and using 4 months in the testing data, but, last, called in different way to using target feature to compare with predicted values only.

Now, the shaping will be the same when your model will be in production, as on testing, because the trained model has specific requirements, such, same shaping (number of columns or dimensions of matrix / vector).

At this point, if my answer is not clear, please detail your doublt again, not the first part, it cover only the framework and environment.

Cheers.