Feature scaling

Question

I am struggling with a conceptual problem related to feature scaling.

Let's assume I am building a classifier (e.g., a NN) and let's assume I rely on future scaling for the input features of my model.

In this context I will normalise the training set using its mean and its std and I would do the same with the testing set using the testing mean and std.

Let us also assume I succeed in building my classifier and I move to production where I try to classify new inputs. However for such new inputs the mean and std are unknown! How can I scale them appropriately before processing with my model? May be I could use the mean and std from training+testing.....

I really don't know which is the correct practice here....any hint?

Thank you for your help!

Answer 1

You should apply the normalization only on your training dataset. Your test set should be kept completely separate and should be used only when your final model has been chosen. If you use include the testing set in the normalization, it can be seen as using the testing set in the training procedure. This is called data snooping.

You should pre-process training dataset and use the obtained mean and std when processing the testing set afterwards. Note that the testing dataset transformations will likely be imperfect (it will not have zero mean or unity standard deviation) but this testing dataset can safely be used since it has not affected any step of the learning process.