3
$\begingroup$

Many people use the mean and variance of the training set to standardize the test set, instead of calculating the mean and variance of the test set and use these. Isn't risky to do that ? If no, why ?

$\endgroup$
2
$\begingroup$

It doesn't make sense to standardize your test set with the mean and variance computed on the test set. At best the test mean and variance will be close enough to the training mean and variance to not break your model - but why not guarantee they are the same by using the same standardization procedure for both. Typically it is best to choose one of the following:

  1. Compute mean and variance on entire data set (train + test) and use these to standardize each set.
  2. Compute mean and variance on train set and use these to standardize each set.

If you are not worried about leaking information from the test set then choose (1), otherwise choose (2).

If you standardize the test set with different values than the training set you might end up with train and test data from different distributions, which will lead to failure to make accurate predictions.

| improve this answer | |
$\endgroup$
1
$\begingroup$

It is not dangerous if you have enough data. If you have enough data, you can somehow estimate the distribution of the phenomenon in hand. If you find the distribution of your sample and its parameters, it means that you know everything about your phenomenon under study. If you are familiar with statistics and probability, you may know that whenever you have enough data, you can estimate the expected value of the random variable using the mean of data samples and standard deviation of the sample will be equal to the standard deviation of the random variable, again if you have enough data. If you have enough data, it means that there will be no difference between the value of mean and expected value also standard deviation of the sample and standard deviation of the population. So if you extract enough data, these values may be so close together. Moreover, the reason we are using standardization of data is that we want to have features with same scale. This is the main reason, consequently there is no need to find the exact value of mean and standard deviation. You may see among machine learning and deep learning practitioners that they may not do this operation because it's a bit time consuming. They just usually divide each feature by the greatest value of the corresponding feature among data samples.

| improve this answer | |
$\endgroup$
  • $\begingroup$ For the CLT, if the number or test samples is < of the number of training sample , the variance of test samples is bigger. So, the variance of the training set and the test set arent equal. How solve this ? $\endgroup$ – Tantaros Feb 3 '18 at 18:29
  • $\begingroup$ @Tantaros surely they won't be equal and there is no need to be equal. We just standardize the data using training samples just for scaling data. The reason is that by doing so we will have data samples which are in same range and by doing so, learning process won't have to learn weights of different ranges. This process is done to accelerate learning process. $\endgroup$ – Media Feb 3 '18 at 18:34
  • $\begingroup$ @Tantaros what does CLT stand for? $\endgroup$ – Media Feb 3 '18 at 18:45
  • $\begingroup$ I'm guessing he means the Central Limit Theorem $\endgroup$ – Imran Feb 4 '18 at 3:57
  • 1
    $\begingroup$ @Media Yes, I agree. $\endgroup$ – Imran Feb 4 '18 at 4:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.