3
$\begingroup$

I want to apply Kmean for clustering after PCA dimensionality reduction. I have standardized data with StandardScaler before the PCA, then I want to train Kmeans for finding clusters. However, the variance among the PCA components could not be of the same order of magnitude.

It is a good practice to standardize the PCA components before the clustering?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

If the variables you are using for k-Means clustering are on different scales the variables with the higher variance will dominate the algorithm, by driving the convergence of the k centroids.

Is this something that you can allow, based on your research goals? If, instead, you want all the factors to have equal weight in the clustering, then you should scale them.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks a lot for the answer! $\endgroup$
    – Galuoises
    Commented Jun 13, 2019 at 9:05
  • $\begingroup$ Is the whiten option in sklearn something that needs to be turned on for the standardisation purpose? $\endgroup$
    – Galuoises
    Commented Apr 25, 2022 at 14:41
  • $\begingroup$ I never used it, but it seems to be doing the same job of the more popular StandardScaler $\endgroup$
    – Leevo
    Commented Apr 25, 2022 at 17:16
2
$\begingroup$

Usually PCA already returns standardized components.

Did you compute the variance of each component? Usually, it will be 1.

The more tricky question is whether to use standardization before doing PCA. I don't think there is a general answer for that.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This is the correct answer. PCA will return a standardized set because the variance (and its magnitude) already went into the eigenvectors. As for the scaling before PCA there's always the case of the wine dataset (i.e. try with and without) $\endgroup$
    – grochmal
    Commented Jun 14, 2019 at 0:33
  • 1
    $\begingroup$ If pca returns standardised components, why there is a whiten parameter in sklearn? $\endgroup$
    – Galuoises
    Commented Apr 25, 2022 at 14:40
  • $\begingroup$ Is there a formal proof that PCA returns a standardized set? Could you please clarify what do you mean with "the variance go in the eigenvectors?" $\endgroup$
    – Galuoises
    Commented Apr 26, 2022 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.