I have studied the usual preprocessing methods for Machine Learning but I couldn't cope the following specific problem.
I apply the "usual" preparation for modeling (dummy variables, normalization, PCA etc., of course in the necessary cases) to the training data. So far, so good. But when I get the to-be-classified new data to make prediction the model constructed above, it's evident that I must apply these preparatory steps to this new data set as well.
And the problem arises here, because if I simply apply the preparatory steps for my new data in turn again, these doesn't take into consideration the characteristics of the training data. So, if I convert the new data factors into dummies, then it takes only the existing factor levels in the new data into account; if I min-max normalize the new data, it will be normalized according its own min-max values, disregarding the values in the training data; and if I use PCA, then the resulting components from the new data will be totally independent of the training data.
So essentially my point is that applying the same conversions separately to the train set and the new data set(s) (which could be only one observation as well), then the two resulting transformed sets will have nothing in common, so the prediction will be baseless.
I found some traces that in some cases there is some "learning" step in the training phase in these transformations as well and apply this "knowledge" to new data (caret and Sklearn, for instance, with "predict" could transform to the new data with characteristics learned from the training data), but generally speaking this inconsistency remains unmentioned otherwise.
What is the correct practice here?