Data scaling before or after PCA

Question

I have seen senior data scientists doing data scaling either before or after applying PCA.

What is more right to do and why?

Answer 1

I once heard a data scinetist state at a conference talk: "Basically, you can do what you want, as long as you know what you are doing."

This also applies here. The more statistically sound way would be to transform all variables prior to additional steps such as PCA or factor analysis. Then you still know the scale of your variables and can interpret the rescaling in the context of your application. If you have no such interpretation, but good reasons for rescaling your principal components due to computational issues arising if some values are to close to zero while others are quite large, rescaling the components makes sense. However, reversing this process and still being able to interpret the effect of the rescaling operation in your context will become almost impossible.

Answer 2

It is definitely recommended to center data before performing PCA since the transformation relies on the data being around the origin. Some data might already follow a standard normal distribution with mean zero and standard deviation of one and so would not have to be scaled before PCA.

If you are getting a number of PCA components for multiple features it is best to scale them as with features of different size, your algorithm might interpret one as more important than others without any real reason.

Answer 3

"It results more important to balance the classes rather than reduce the dimensionality, at least in terms of accuracy; (ii) The best choice seems to be the application of SMOTE followed by PCA.."

Link: https://core.ac.uk/download/pdf/61408511.pdf