0
$\begingroup$

Given the coefficients of PC1 as follows for each variable (0.30, 0.31, 0.42, 0.37, 0.13, -0.43, 0.29, -0.42, -0.11) which variables contributes most to this PC? Does the sign(+/-) matters or considering the absolute value is enough?

$\endgroup$
0
$\begingroup$

Welcome to the site. PCA is an unsupervised dimensionality reduction algorithm. It works by transforming the original feature-set into eigen-vectors that are difficult to map with the original feature set. As such, the first Principal Component (PC) contains the features with maximum variance. The subsequent PCs contain features with decreased variance to the first PC.

With this background, I invite you to read this Q on SO. It has the solution to programmatically determine the features deemed most important by PCA.

[edited] Regarding the sign of the components, eve if you change them you do not change the variance that is contained in the first component. Moreover, when you change the signs, the weights (prcomp( ... )$rotation) also change the sign, so the interpretation stays exactly the same:

set.seed( 2020 )
df <- data.frame(1:10,rnorm(10))
pca1 <- prcomp( df )
pca2 <- princomp( df )
pca1$rotation

gives

                PC1        PC2
X1.10     0.9876877  0.1564384
rnorm.10. 0.1564384 -0.9876877

and pca2$loadigs gives,

               Comp.1 Comp.2
SS loadings       1.0    1.0
Proportion Var    0.5    0.5
Cumulative Var    0.5    1.0

Then the question arises that why the interpretation remains the same

You do the PCA regression of y on component 1. In the first version (prcomp), say the coefficient is positive: the larger the component 1, the larger the y. What does it mean when it comes to the original variables? Since the weight of the variable 1 (1:10 in df) is positive, that shows that the larger the variable 1, the larger the y.

Now use the second version (princomp). Since the component has the sign changed, the larger the y, the smaller the component 1 -- the coefficient of y< over PC1 is now negative. But so is the loading of the variable 1; that means, the larger variable 1, the smaller the component 1, the larger y -- the interpretation is the same.

The conclusion is that for each PCA component, the sign of its scores and of its loadings is arbitrary and meaningless. It can be flipped, but only if the sign of both scores and loadings is reversed at the same time.

Furthermore, the directions that the principal components act correspond to the eigenvectors of the system. If you are getting a positive or negative PC it just means that you are projecting on an eigenvector that is pointing in one direction or 180∘ away in the other direction. Regardless, the interpretation remains the same! It should also be added that the lengths of your principal components are simply the eigenvalues.

$\endgroup$
5
  • $\begingroup$ I get the point that the 1st PC contains the maximum variance. My question is how to find out which variables contribute most to a particular PC by looking at the coefficient values. Considering the highest absolute values is correct or the (+/-) sign matters? $\endgroup$ – Subash Basnayake Oct 6 '20 at 1:39
  • $\begingroup$ Did you even care to read the link I posted to my answer? Because if you had, this Q that your now asking should not have arisen!!! $\endgroup$ – mnm Oct 6 '20 at 2:36
  • $\begingroup$ Of course, I went through that link. But I am still struggling to find an answer for my Q inside that. Apologies if I am missing something. I'm totally new to this PCA. I have seen that the highest values are contributing the most but when there are both + and - coefficients what should be selected? Actually, this is a question on a paper and we are confused about what the answer is. $\endgroup$ – Subash Basnayake Oct 6 '20 at 3:48
  • $\begingroup$ @SubashBasnayake I have edited my answer. Hope this helps to aid your understanding. $\endgroup$ – mnm Oct 6 '20 at 4:39
  • $\begingroup$ Thank you @mnm for the explanation. $\endgroup$ – Subash Basnayake Oct 7 '20 at 0:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.