I have serious doubts concerning the features standardization done before the learning process of a multilayer perceptron.
I'm using python-3 and the scikit-learn package for the learning process and for the features normalization.
As suggested from the scikit-learn wiki (Tips on pratical use), I'm doing a features standardization with the Preprocessing module, which means that all my features will appear as standard normal specifications.
The problem is that in the showed example:
from sklearn.preprocessing import StandardScaler scaler = StandardScaler() # Don't cheat - fit only on training data scaler.fit(X_train) X_train = scaler.transform(X_train) # apply same transformation to test data X_test = scaler.transform(X_test)
The standardization is done first on the training set, and then, after that, it's applied on the test set.
Which means, that the distributions parameters (mean, st.deviation) will be estimated only from the training set.
Now, because I need to save my multiple datasets in a database, on which I will perform the hold-out sampling for the training-test folds, I have done a preprocessing on all my instances at once, because I don't want to do that at run time execution.
It's clear that in this way the distributions parameters will change and the results will be different.
Now, Someone knows the differences between the two approaches? Are there issues doing the preprocessing at once on all the dataset?