I understand that one should standardize and normalize the test data (or any "unlabeled" data) with the training mean and sd. How can I implement this in R language? Is there a kind of "fitting" to the training set and a kind of applying to the test data?
Check out the
preProcess function from the
caret library. You can choose the parameters you want to scale/center the training data, and it also saves the transformations it makes so then you can normalize the test set with the same specifications that you normalized the training set with. Could go something like this:
library(caret) trainData <- data.frame(v1 = rnorm(15,3,1), v2 = rnorm(15,2,2)) testData <- data.frame(v1 = rnorm(5,3,1), v2 = rnorm(5,2,2)) normParam <- preProcess(trainData) norm.testData <- predict(normParam, testData)
norm.testData is scaled and centered according to the training data set parameters.
Another way to do this without using caret:
## set up data trainData <- data.frame(v1 = rnorm(15,3,1), v2 = rnorm(15,2,2)) testData <- data.frame(v1 = rnorm(5,3,1), v2 = rnorm(5,2,2)) ## find mean and sd column-wise of training data trainMean <- apply(trainData,2,mean) trainSd <- apply(trainData,2,sd) ## centered sweep(trainData, 2L, trainMean) # using the default "-" to subtract mean column-wise ## centered AND scaled norm2.testData <- sweep(sweep(testData, 2L, trainMean), 2, trainSd, "/")