# Eigen Decomposition of Data Matrix for PCA

In PCA we Eigen decompose the covariance matrix, not data matrix, Is it because most data matrices are non-square. If they were, isn't is correct to eigen decompose data matrix than the covariance matrix?

## Some Intro

Eigen-Decompostion does lots of things among them one thing is interesting. Eigen-Decomposition usually captures the characteristic of the matrix it is applied to. For instance is the matrix is the matrix of similarities, the result of eigen decomposition will be a clustering (see Spectral Clustering).

## PCA

The idea behind PCA is to find coordinates on which the intrinsic variance of data is maximized. Well ... according to Intro, if I have a matrix encoding joint variance (co-variance) of all pairs of variables, then I can use eigen decomposition to simply capture this property for me! This is what PCA does.

• If my data matrix is square and If I do Eigen decomposition on it, and take the top eigen values and vectors, will it be equivalent to pca? – tam Jul 10 '19 at 11:14
• NO! Not only "no" but also the question is wrong! when a mathematical approach is right, then it is right in general! Imagine i have age and salary of 2 people. It is a squared matrix and according to you the eigen-decomposition is fine. Now I suddenly get the hight of these people as well and my matrix becomes 2x3 so eigen-decomposition is not valid. Now, isn't it obviously wrong is u choose a math approach which goes from valid to invalid by obtaining height of your samples?!!! – Kasra Manshaei Jul 10 '19 at 11:22
• That is conceptually strange (i never did it to be honest. Create a squared data with two obvious clusters and apply eigen-decomposition and see what happens. I never did) – Kasra Manshaei Jul 10 '19 at 11:23
• I was just wondering whether eigen decomposition do the work of dimensionlity reduction in the case of ONLY SQUARE MATRIX. If it does, then will it be as good as pca. – tam Jul 10 '19 at 11:31
• Thanks for your reply – tam Jul 10 '19 at 11:32