Intuition behind the PCA algorithm

Question

I am trying to understand PCA intuitively. Here it goes: After finding the eigenvectors and eigenvalues of the covariance matrix of the dataset, the eigenvalues will represent how spread out the dataset is, and the eigenvector will represent the direction of the eigenvalue. And what we are trying to do is to order the eigenvector from highest eigenvalue to lowest (highest meaning the features are more spread out hence more independent of each other and less redundant) and project the original dataset onto the new set of "axis" (the eigenvectors). For example, if we have a 3-D dataset, then there will be three eigenvectors each orthogonal to each other (so they will form a 3-D space). Next, we project the original dataset onto this new set of "axis" space, and if we want to reduce to 2-D, we would take out the least significant "axis" (smallest eigenvalue)

I am familiar with the technical steps of performing PCA, but I am struggling to understand the actual intuition behind the algorithm. Were there any mistakes or flaws in my explanation above, especially the part of explaining dimension reduction with the use of the word 'axis'? Thanks a lot in advance!

Answer 1

In simple words, PCA (Principal Component Analysis) is a dimension-reduction method where you try to reduce the number of variables from a larger number to a smaller one without any loss of the important information available in the data. Mathematically, it is a projection of a higher-dimensional object in a lower-dimensional vector space.

A great example in a real-life scenario is where we see 3D objects in a 2D Television. Reference

Another good explanation on the intuition on PCA

Hope it helps.