In PCA we Eigen decompose the covariance matrix, not data matrix, Is it because most data matrices are non-square. If they were, isn't is correct to eigen decompose data matrix than the covariance matrix?


Some Intro

Eigen-Decompostion does lots of things among them one thing is interesting. Eigen-Decomposition usually captures the characteristic of the matrix it is applied to. For instance is the matrix is the matrix of similarities, the result of eigen decomposition will be a clustering (see Spectral Clustering).


The idea behind PCA is to find coordinates on which the intrinsic variance of data is maximized. Well ... according to Intro, if I have a matrix encoding joint variance (co-variance) of all pairs of variables, then I can use eigen decomposition to simply capture this property for me! This is what PCA does.

  • $\begingroup$ If my data matrix is square and If I do Eigen decomposition on it, and take the top eigen values and vectors, will it be equivalent to pca? $\endgroup$ – tam Jul 10 '19 at 11:14
  • 1
    $\begingroup$ NO! Not only "no" but also the question is wrong! when a mathematical approach is right, then it is right in general! Imagine i have age and salary of 2 people. It is a squared matrix and according to you the eigen-decomposition is fine. Now I suddenly get the hight of these people as well and my matrix becomes 2x3 so eigen-decomposition is not valid. Now, isn't it obviously wrong is u choose a math approach which goes from valid to invalid by obtaining height of your samples?!!! $\endgroup$ – Kasra Manshaei Jul 10 '19 at 11:22
  • $\begingroup$ That is conceptually strange (i never did it to be honest. Create a squared data with two obvious clusters and apply eigen-decomposition and see what happens. I never did) $\endgroup$ – Kasra Manshaei Jul 10 '19 at 11:23
  • $\begingroup$ I was just wondering whether eigen decomposition do the work of dimensionlity reduction in the case of ONLY SQUARE MATRIX. If it does, then will it be as good as pca. $\endgroup$ – tam Jul 10 '19 at 11:31
  • $\begingroup$ Thanks for your reply $\endgroup$ – tam Jul 10 '19 at 11:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.