2
$\begingroup$

Is there any reason to think that SVD is better than PCA (by eigendecomposition) in decorrelating the predictors of a machine learning model?

$\endgroup$
1
  • $\begingroup$ @BrianK, thanks for your comment. 1. I find this answer pretty witty but not very serious. To start with, when I am talking about PCA then I am talking about PCA with eigendecomposition. When we are talking in a short way about PCA then we usually mean this. 2. That's a very nice answer it does not directly answer my question. So let's please stay on topic: any specific answer to my question? $\endgroup$
    – Outcast
    Oct 26, 2018 at 9:10

1 Answer 1

1
$\begingroup$

To the best of my knowledge, the answer to your question is no. Regarding finding the correlations of different variables, they work the same. They both capture linear associations and do not capture nonlinear ones. The difference between them is mostly about numerical computation which makes SVD more handy than traditional PCA. I recommend having a look at this answer and this explanation.

As a final remark, let’s discuss the numerical advantages of using SVD. A basic approach to actually calculating PCA on a computer would be to perform the eigenvalue decomposition of $X^TX$ directly. It turns out that doing so would introduce some potentially serious numerical issues that could be avoided by using SVD.

$\endgroup$
2
  • $\begingroup$ Thanks for your good answer(upvote). I also really like the article at the second link which you posted. $\endgroup$
    – Outcast
    Oct 26, 2018 at 12:32
  • $\begingroup$ I am glad it helped :) Good Luck! $\endgroup$ Oct 26, 2018 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.