1
$\begingroup$

I have a question related to K-Means clustering and PCA. In my project, I have two target classes - 0 and 1- and I am trying to group the records that were predicted as 0 into 5 clusters. I am using PCA strictly as a visualization technique since my data frame has 8 dimensions and I need to bring it down to 2-3 dimensions to see the clusters. My question is about the procedure I should follow~

First Way:

  1. Extract all records with target = 0 
  2. Do PCA and KMeans on just those records

Second way:

  1. Do PCA on all records (target = 0 and 1) 
  2. Extract PCA records with target = 0 (from the PCA data frame created in step 1)
  3. Do KMeans on those records

The PCA1, PCA2, PCA3 values for the records(with target = 0) are different using these two ways. And since the PCA values are different, the cluster visualizations are also showing up differently. Which option should I follow?

Thank you so much!

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ What is your end goal/objective. What one thing you would like to know from your data? $\endgroup$
    – 10xAI
    Commented Jul 9, 2020 at 2:30
  • 1
    $\begingroup$ The two approaches produce different results, in general. However both approaches are correct when they reflect your actual priorities with the dataset at hand. There is no generaly correct answer without specifying the requirements you try to fullfill and prioritise. Then you can choose which approach pf the two reflects your priorities with the data $\endgroup$
    – Nikos M.
    Commented Jul 9, 2020 at 16:51
  • $\begingroup$ My end goal is just to see how cleanly grouped the 5 clusters of target = 0 are. They are supposed to signify stages of a disease so I would like to know which point might be in which stage etc ~ $\endgroup$
    – moii789
    Commented Jul 9, 2020 at 19:18
  • $\begingroup$ Ok, since you want to see the important features of one class only, then my opinion is that doing PCA and kmeans on only records of that class would get you the important information for this class without being polluted by information from other classes. This is similar to the case where one already has the data of a single class and wants to explore it, without any extra assumptions about other possibly unrelated cases $\endgroup$
    – Nikos M.
    Commented Jul 10, 2020 at 9:49

1 Answer 1

Reset to default
2
$\begingroup$

My answer would be second option

I think the use of PCA is to represent original high dimensional information/data in lower dimension by calculationg the direction/axes along which there is maximum variablity in data.

In first case, where you filter for 0-labeled observations and then do PCA so PCA would measure variablity based on a smaller version of original data, and would make different axes than the second case, where the PCA would measure variablity accross entire data and hence the axes made in second case could be different. So in the first option the dataset you get after PCA is not a correct representation of high dimensional 0-labeled observations.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.