In PCA we Eigen decompose the covariance matrix, not data matrix, Is it because most data matrices are non-square. If they were, isn't is correct to eigen decompose data matrix than the covariance matrix?
1 Answer
$\begingroup$
$\endgroup$
4
Some Intro
Eigen-Decompostion does lots of things among them one thing is interesting. Eigen-Decomposition usually captures the characteristic of the matrix it is applied to. For instance is the matrix is the matrix of similarities, the result of eigen decomposition will be a clustering (see Spectral Clustering).
PCA
The idea behind PCA is to find coordinates on which the intrinsic variance of data is maximized. Well ... according to Intro, if I have a matrix encoding joint variance (co-variance) of all pairs of variables, then I can use eigen decomposition to simply capture this property for me! This is what PCA does.
answered Jul 10, 2019 at 11:10
-
$\begingroup$ If my data matrix is square and If I do Eigen decomposition on it, and take the top eigen values and vectors, will it be equivalent to pca? $\endgroup$– tamCommented Jul 10, 2019 at 11:14
-
1$\begingroup$ NO! Not only "no" but also the question is wrong! when a mathematical approach is right, then it is right in general! Imagine i have age and salary of 2 people. It is a squared matrix and according to you the eigen-decomposition is fine. Now I suddenly get the hight of these people as well and my matrix becomes 2x3 so eigen-decomposition is not valid. Now, isn't it obviously wrong is u choose a math approach which goes from valid to invalid by obtaining height of your samples?!!! $\endgroup$ Commented Jul 10, 2019 at 11:22
-
$\begingroup$ That is conceptually strange (i never did it to be honest. Create a squared data with two obvious clusters and apply eigen-decomposition and see what happens. I never did) $\endgroup$ Commented Jul 10, 2019 at 11:23
-
$\begingroup$ I was just wondering whether eigen decomposition do the work of dimensionlity reduction in the case of ONLY SQUARE MATRIX. If it does, then will it be as good as pca. $\endgroup$– tamCommented Jul 10, 2019 at 11:31