Should one apply dimensionality reduction methods to the data set before or after train-test splitting? Anyway, in case of training a model with preprocessing by dim-red, one should apply the same dim-red to the future instances to be predicted, right? But how could you reduce the dimensionality of one single instance in the same way?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
It depends on the type of algortithm you use for dimensionality reductions. In case you use PCA, you should build your PCA on your train set. Then you need to set your principal components to transform your points in test set into the same space. This way you can then use train and test set in the same reducted space.
-
$\begingroup$ But how could you transform the test set to the new feature space? In R, for instance,
prcompsimply calculates the synthetic features of the given data set, you cannot instruct it to calculate these on the basis of any previous components or data set, can you? $\endgroup$– HendrikCommented Aug 23, 2016 at 13:50 -
1$\begingroup$ you can use the
predict(prcomp object, newdata). See?predict.prcomp$\endgroup$ Commented Aug 23, 2016 at 19:24
$\begingroup$
$\endgroup$
You should always use just your training data and then apply the same transformation on your test data. This is a much fairer representation of how your model will perform when real new unseen samples are fed to your model. The point of a test set is trying to estimate generalization on unseen data, using your test data leaks information into your training set. Plus, if you cannot apply the transformation to your test data, how will you apply it to new data?
answered Aug 23, 2016 at 11:50