Many people use the mean and variance of the training set to standardize the test set, instead of calculating the mean and variance of the test set and use these. Isn't risky to do that ? If no, why ?
It doesn't make sense to standardize your test set with the mean and variance computed on the test set. At best the test mean and variance will be close enough to the training mean and variance to not break your model - but why not guarantee they are the same by using the same standardization procedure for both. Typically it is best to choose one of the following:
- Compute mean and variance on entire data set (train + test) and use these to standardize each set.
- Compute mean and variance on train set and use these to standardize each set.
If you are not worried about leaking information from the test set then choose (1), otherwise choose (2).
If you standardize the test set with different values than the training set you might end up with train and test data from different distributions, which will lead to failure to make accurate predictions.
It is not dangerous if you have enough data. If you have enough data, you can somehow estimate the distribution of the phenomenon in hand. If you find the distribution of your sample and its parameters, it means that you know everything about your phenomenon under study. If you are familiar with statistics and probability, you may know that whenever you have enough data, you can estimate the expected value of the random variable using the mean of data samples and standard deviation of the sample will be equal to the standard deviation of the random variable, again if you have enough data. If you have enough data, it means that there will be no difference between the value of mean and expected value also standard deviation of the sample and standard deviation of the population. So if you extract enough data, these values may be so close together. Moreover, the reason we are using standardization of data is that we want to have features with same scale. This is the main reason, consequently there is no need to find the exact value of mean and standard deviation. You may see among machine learning and deep learning practitioners that they may not do this operation because it's a bit time consuming. They just usually divide each feature by the greatest value of the corresponding feature among data samples.