Is there any reason to think that SVD is better than PCA (by eigendecomposition) in decorrelating the predictors of a machine learning model?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I'm leaving this as an answer because I don't have enough reputation here to comment: 1. As elucidated in the accepted answer to this same question over at stats, the question doesn't quite make sense. 2. I recommend checking out this very useful answer that should clear out some further confusions you may have about PCA, SVD and their connection. $\endgroup$– Brian KOct 25, 2018 at 17:05
-
$\begingroup$ @BrianK, thanks for your comment. 1. I find this answer pretty witty but not very serious. To start with, when I am talking about PCA then I am talking about PCA with eigendecomposition. When we are talking in a short way about PCA then we usually mean this. 2. That's a very nice answer it does not directly answer my question. So let's please stay on topic: any specific answer to my question? $\endgroup$– OutcastOct 26, 2018 at 9:10
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
To the best of my knowledge, the answer to your question is no. Regarding finding the correlations of different variables, they work the same. They both capture linear associations and do not capture nonlinear ones. The difference between them is mostly about numerical computation which makes SVD more handy than traditional PCA. I recommend having a look at this answer and this explanation.
As a final remark, let’s discuss the numerical advantages of using SVD. A basic approach to actually calculating PCA on a computer would be to perform the eigenvalue decomposition of $X^TX$ directly. It turns out that doing so would introduce some potentially serious numerical issues that could be avoided by using SVD.
answered Oct 26, 2018 at 12:25
-
$\begingroup$ Thanks for your good answer(upvote). I also really like the article at the second link which you posted. $\endgroup$– OutcastOct 26, 2018 at 12:32
-