K means visualisation after reducing dimensionality with PCA

Question

In clustering ($K$ means, for example) when I have $N$ features and after creating the model (with this $N$ features) to visualize this model I need to reduce this $N$ dimensions into $2$ or $3$ dimensions, let's say I will use (PCA) fro example.

My question is how I can analyze the results (grape with principal component)?

this is simple example :

Data without reducing : AGE,GENDER,SPENT,SALARY,CAR,.....

Data after reducing dimension:

Principal Component $1$, Principal Component $2$, Principal Component 3

what do PC1,PC2,PC3 means

Answer 1

K-means minimizes sum of squared errors, and PCA finds the projection with maximum sum of squares. So they are a quite natural fit.

Just run k-means and project it to 2d for visualization with PCA. They you are largely seeing the data the same way as k-means (if you only use the rotation, not the scaling!)

What I'd be more concerned about is the input data, as it is not particularly well suited for neither k-means nor PCA. So I wouldn't be surprised if the results are barely interpretable. Both methods make most sense if the input variables are continuous and of the same scale.

Answer 2

The principal components are directions that are orthogonal to each other and the first $3$ components should carry the most variance with them.

To obtain them, you project the data onto the $3$ leading eigenvectors of the covariance matrix.

You can then perform $k$-means on the projected data and color each group with different color to facilitate visualization.

Answer 3

Principal components are the results of projections of your features on 3 directions that carry a lot of a variance and a minimum of information loss ( when you take a projection and reduce it from 3-D to 2-D for example, you lose a bit of information ). You could say ,in other words, that PC 1, PC 2 , PC 3 are somewhat a combination of all your features you used for PCA.