1
$\begingroup$

I have a dataset of shape (29088, 11). When I apply the Kmeans where K=2 I get the following plot:

enter image description here

I am surprised that the value of Sum Squared Error (SSE) for C0 (in blue) is smaller than the value of SSE for C1 (in red). Isn't supposed to be the opposite as is demonstrated in the plot where the blue points are distorted which means the value of SSE should be larger?

Note: C0 has 8554 points (in blue) while C1 has 20534 points (in red)

$\endgroup$

1 Answer 1

1
$\begingroup$

I believe that the number of elements in C1 clusters are more than that of C0. Can you please check that once?

C0 has 8554 samples, thus the average SSE becomes $\frac{28101.1}{8544} = 3.28$. While C1 contains 20534 points with average SSE of $\frac{47725.5}{20534}=2.324$.

This implies that the C1 cluster is more contained, it has a very high SSE because it contains more than 2x times the points present in C0.

$\endgroup$
6
  • $\begingroup$ Yes, this is correct. You can see that in the plot's legend. For the sake of completeness, C0 has 8554 points (in blue) while C1 has 20534 points (in red) $\endgroup$
    – Dave
    Apr 17, 2022 at 21:50
  • $\begingroup$ The mean SSE of C0 (3.28 ) is greater than that of C1( 2.32), as expected. $\endgroup$ Apr 18, 2022 at 15:47
  • $\begingroup$ Kindly can you elaborate on why the mean of C0 is higher whereas it has fewer samples than C1? Looking at the plot, C0 (in blue) is noticeably distorted which does not make sense having lower SSE than the densely C1. $\endgroup$
    – Dave
    Apr 18, 2022 at 15:58
  • 1
    $\begingroup$ I have updated my answer, please have a look at it. lemme know if still unclear $\endgroup$ Apr 18, 2022 at 16:05
  • $\begingroup$ Just a quick comment. The plot is after a PCA (or some other dimension reduction technique), right? If you are just projecting in two dimensions, then it is unusual to see a line separating the two clusters. If that is the case, then I believe the clustering takes into account only 2 dimensions... $\endgroup$ Apr 21, 2022 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.