1
$\begingroup$

It is stated that for: Feature Normalization -

The test set must use identical scaling to the training set.

And the point is given that:

Do not scale the training and test sets using different scalars: this could lead to random skew in the data.

Could someone explain what that means?

$\endgroup$
4
$\begingroup$

Generally speaking, best practice is to use only the training set to figure out how to scale / normalize, then blindly apply the same transform to the test set.

For example, say you're going to normalize the data by removing the mean and dividing out the variance. If you use the whole dataset to figure out the feature mean and variance, you're using knowledge about the distribution of the test set to set the scale of the training set - 'leaking' information.

The right way to do this is to use only the training set to calculate the mean and variance, normalize the training set, and then at test time, use that same (training) mean and variance to normalize the test set.

As for the point in your question, imagine using the training mean and variance to scale the training set and test mean and variance to scale the test set. Then, for example, a single test example with a value of 1.0 in a particular feature would have a different original value than a training example with a value of 1.0 (because they were scaled differently), but would be treated identically by the model. This is where the bias would come from.

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.