# Alternative methods for improved clustering separation?

I have the following labeled cluster, which is what an ideal clustering algorithm would generate:

Now, I have applied a basic K-Means clustering algorithm to the data, and the outcome is as follows:

I recognize that this is a tough problem to properly cluster because some of the classes are so similar.

But I was wondering if there are any alternative algorithms that may help me improve the separability of the clusters, and improve how well my unsupervised clustering algorithm would work on new data?

• Try semi-supervised learning, since you have training data, and manifold learning.
– Emre
Mar 10, 2017 at 22:37

Your data doesn't appear to be easily separable. In general, one could apply some kind of transformation that pulls apart the distributions for each class. Having labels available makes it possible, in principle, to learn such a transformation (as @Emre metnioned in the comments). But, there are a couple issues with your particular data set. 1) You don't appear to have many data points (unless you've only plotted a small subset). This would limit you to very simple transformations (otherwise you'd probably get severe overfitting). 2) The points are simply overlapping. A transformation can only work based on its inputs and, if the coordinates are indistinguishable, there's nothing that can be done. In the best case, you might be able to pull the lower left turquoise cluster and the yellow points further from the main mass, but the rest of the points are pretty much intermingled. Any transformation that could manage to separate them in the training data would be very complicated, and probably just reflect sample noise (i.e. it would probably be completely overfit, and not generalize to new data).

The ideal thing would be to find/measure additional (relevant) variables. In this case, the classes may become separable in the higher dimensional space. For example, imagine additing a third axis, where the red points become 'lifted' above the blue points.

You forgot the most important step.

Preprocessing.

Look at the axes. Scale them the same way, and you will realize that your y axis has zero effect. Your data really looks like this squeezed slice:

Don't blindly run clustering. Your work is likely 70% figuring out how to adequately preprocess your data, 10% clustering, and 20% making sense of the outcome.

Don't underestimate how difficult preprocessing is. The way you used k-means, you assumed 1 gallon of water = 1 second. That assumption is probably wrong...

Last but not least: since you have labels, why use clustering at all?

• Thanks for your response! Your attached photo appears to be warped. Could you re-upload it?
– Gary
Mar 19, 2017 at 16:58
• @Gary no, if deliberately is that way, because that is what your data looks like! I just set the aspect ratio on your image to better match your scale. Mar 19, 2017 at 19:15

Clustering cannot inherently extract labelled classes. If you have labels then you should use those with a supervised algorithm. There is no reason that any clustering should agree with pre-provided class labels. Imagine a data set that was perfectly uniformly distributed. One could have class labels for that data that can be quite arbitrary (you could even restrict to arbitrary convex regions). What clustering result should you expect for perfectly uniformly distributed data? Is there any reason that should ever be able to match any particular class labelling other than by chance? Clustering corresponds to the something about the distributional properties of your data set. Unless the class labels happen to align with those distributional properties (and there is no reason that they should) then clustering cannot recover the labels.