2
$\begingroup$

I'm reding Introduction to Statistical Learning (ISL), Ch. 6 Principle Component Regression. When I look at the Labs, I wonder how scaling is handled in R in this case.

The ISL-Lab "Chapter 6 Lab 3: PCR and PLS Regression" suggests the following approach:

pcr.fit=pcr(Salary~., data=Hitters,subset=train,scale=TRUE, validation="CV")
pcr.pred=predict(pcr.fit,x[test,],ncomp=7)

Here train and test are lists of row indexes, indicating test and training data, like:

> train
  [1]  70  98 150 237  53 232 243

Question:

During training the data is scaled by scale=TRUE. However, scaling appears to be only used for subset=train if I'm not wrong (which seems correct).

However, when predicting on x[test,], no scaling seems to take place. My take would be that the same scaling should be applied to the test data as was used to scale the training data in order to produce correct predictions.

Am I wrong here? Or is there anything going on in the background with pcr() which makes scaling the test set unnecessary?

Unfortunately the DOCS come with no details on scale() with pcr().

What is the correct approach here?

$\endgroup$
3
$\begingroup$

I agree that if scaling is used on the training data, it should also be used on the test data.

However, from what I see in the pcr documentation there is not scale option in the function. This seems to be confirmed by running the code from the example in the documentation: https://www.rdocumentation.org/packages/analogue/versions/0.17-5/topics/pcr

> library(analogue)
> data(ImbrieKipp)
> data(SumSST)
> pcr(ImbrieKipp, SumSST, tranFun = Hellinger, scale = TRUE)

    Principal Component Regression Model

Call:
pcr(x = ImbrieKipp, y = SumSST, tranFun = Hellinger, scale = TRUE)

No. of Components: 27

RMSE (Apparent):
     PC1      PC2      PC3      PC4      PC5      PC6      PC7      PC8      PC9     PC10     PC11 
2.381215 1.707588 1.680896 1.679774 1.608903 1.535145 1.507995 1.496939 1.496908 1.432142 1.426444 
    PC12     PC13     PC14     PC15     PC16     PC17     PC18     PC19     PC20     PC21     PC22 
1.405155 1.391348 1.349172 1.349172 1.315284 1.313187 1.311801 1.291201 1.206484 1.188438 1.187503 
    PC23     PC24     PC25     PC26     PC27 
1.171215 1.170947 1.170380 1.162497 1.162355 
> pcr(ImbrieKipp, SumSST, tranFun = Hellinger, scale = FALSE)

    Principal Component Regression Model

Call:
pcr(x = ImbrieKipp, y = SumSST, tranFun = Hellinger, scale = FALSE)

No. of Components: 27

RMSE (Apparent):
     PC1      PC2      PC3      PC4      PC5      PC6      PC7      PC8      PC9     PC10     PC11 
2.381215 1.707588 1.680896 1.679774 1.608903 1.535145 1.507995 1.496939 1.496908 1.432142 1.426444 
    PC12     PC13     PC14     PC15     PC16     PC17     PC18     PC19     PC20     PC21     PC22 
1.405155 1.391348 1.349172 1.349172 1.315284 1.313187 1.311801 1.291201 1.206484 1.188438 1.187503 
    PC23     PC24     PC25     PC26     PC27 
1.171215 1.170947 1.170380 1.162497 1.162355 

I did check the the data were not standardised to begin with !

So in your case I would simply use scale() on the data before running pcr


Edit to address the point raised in comments. In the pcr function there is no warning or error if you use a completely fictitious variable:

> pcr(ImbrieKipp, SumSST, tranFun = Hellinger, SomethingIjustMadeUp = TRUE)

    Principal Component Regression Model

Call:
pcr(x = ImbrieKipp, y = SumSST, tranFun = Hellinger, SomethingIjustMadeUp = TRUE)

No. of Components: 27

RMSE (Apparent):
     PC1      PC2      PC3      PC4      PC5      PC6      PC7      PC8      PC9     PC10     PC11 
2.381215 1.707588 1.680896 1.679774 1.608903 1.535145 1.507995 1.496939 1.496908 1.432142 1.426444 
    PC12     PC13     PC14     PC15     PC16     PC17     PC18     PC19     PC20     PC21     PC22 
1.405155 1.391348 1.349172 1.349172 1.315284 1.313187 1.311801 1.291201 1.206484 1.188438 1.187503 
    PC23     PC24     PC25     PC26     PC27 
1.171215 1.170947 1.170380 1.162497 1.162355 

No warning or error

| improve this answer | |
$\endgroup$
  • $\begingroup$ Well, yes I think this makes sense. However, I wonder why there is a scale option in PCR at all. $\endgroup$ – Peter Sep 5 at 13:22
  • $\begingroup$ But that's what I'm saying, there doesn't seem to be a scale option. It's not in the documentation and using it appears to do nothing. You can put ` foobar_qwerty = TRUE` in there too, and that does nothing either. $\endgroup$ – Robert Long Sep 5 at 14:12
  • $\begingroup$ I see that there is no such options in the docs. But still the code is working (no error). So it is unclear to me what's behind it. Also the code for ISL - quite a famous book - seems to be a little dodgy in Ch. 6 $\endgroup$ – Peter Sep 5 at 14:18
  • $\begingroup$ But like I just said, you can put any variable you want in there and it doesn't generate an error or warning. See the edit I just added to the answer $\endgroup$ – Robert Long Sep 5 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.