# Does PCA change the values of the data?

Principal Component Analysis is a means to reduce the dimensionality of data, if I understand correctly.

So if I have a 1000 sample point 12 dimensional matrix and reduce it to a 1000 sample point 2 dimensional one, then are the values of the sample points themselves changed in some way? or are simply 10 dimensions/columns thrown out and 2 of the original remain?

• Welcome to DataScience.SE! You are running into the difference between feature selection (selecting a subset of the original features) and feature extraction (deriving new ones).
– Emre
Jul 7, 2016 at 17:33
• Glad to be here! So PCA falls into feature selection right? Because feature extraction varies from field by field,application to application, if I'm not wrong? Jul 8, 2016 at 23:46
• No, feature extraction, because the principal components returned by PCA are functions of the input features, not usually a subset of them. I think this will become more clear if you apply PCA to some data and see what the principal components are.
– Emre
Jul 8, 2016 at 23:48
• @Emre Ok. So PCA is feaure extraction? Wow, I guess you learn something new everyday! Jul 9, 2016 at 8:41
• – smci
Dec 13, 2016 at 0:36

Yes, the new 2 dimensional values will be a projection of original 12 dimensional points onto the two principle components (vectors).

please refer to the first figure in this clear tutorial: http://lazyprogrammer.me/tutorial-principal-components-analysis-pca/

• I have a bad habit, but I like to confirm things, so you're saying PCA does in fact changes values of data right? Does this ever cause any significant loss ever? Jul 8, 2016 at 23:48
• Of course it can cause significant loss. Any dimensionality reduction can cause loss, or we would always reduce data to 0 dimensions. Jul 9, 2016 at 1:42
• @Anony-Mousse if we did that then there would be no data to learn from, and reducing too many dimensions may lead to subsequent overfitting. Jul 9, 2016 at 8:38
• Therefore, every dimensionality reduction will be lossy at some point (or not reduce dimensionality anymore). Proof complete. Jul 9, 2016 at 9:39

PCA is a transform: it creates new (transformed) features from the original data. In general if you choose fewer dimensions (e.g. you chose to reduce m=12 -> n=2 dimensions), it's lossy and will throw away some of in the information content of the original data. The higher n is, the less you lose, and for m=n, you preserve all the original information (although you still do a vector transform on the data, so the extracted features are != the original data).

It was your (arbitrary) decision to choose the parameter n=2 (number of Principal Components), you could try other values or explore a range. You could have chosen n=5, n=9, or even the maximum possible: n=12.

For standard rules-of-thumb on how to choose n, see e.g. Choosing number of principal components to retain

(Scree plot, Proportion of total variance explained, Average eigenvalue rule, Log-eigenvalue diagram, etc.)

where a Scree Plot is a simple line-segment plot that shows the fraction of total variance in the data as explained or represented by each PC. Usually the scree plot will have a knee where the number of PCs explains most of the variance, and if so that might suggest you an upper bound on n.

There are other rules-of-thumb discussed there too. You can find tons of articles on this subject.