I have the following time series data set
Each row is a unique Item, and each column shows the amount purchased per day. There are a total of 33 columns.
I'm taking the first 32 columns(leaving out the last column, which will be my target) as my training set, and the last 32 rows (leaving out the first column) as my testing set
X_train = dataset[:, :-1] # taking all columns except the last column y_train = dataset[:, -1:] # setting the last column to be the target X_test = dataset[:, 1:] # taking all columns expect the first column
I'm going to feed
y_train in my LSTM model, and use the model to perform prediction on
Now, I wish to performing minmax scaling on
dataset before performing training, but I have some questions:
from sklearn.preprocessing import MinMaxScaler scaler = MinMaxScaler(feature_range=(-1, 1)) scaled_dataset = scaler.fit_transform(dataset)
This has obvious data leakages, because the scaler is fitted with the test values.
I thought of creating 2 scalers, one for the training set, and one for the target
scaler_x = MinMaxScaler(feature_range=(-1, 1)) scaler_y = MinMaxScaler(feature_range=(-1, 1)) scaled_all_data = scaler_x.fit_transform(dataset[:, :-1]) scaled_y = scaler_y.fit_transform(dataset[:, -1:])
I'm not sure if that's the right approach. I've already tried searching for answers, but their situation is not quite like mine, or the questions are unanswered yet.
Any advice on how I should perform value scaling?