5
$\begingroup$

since some time I have a question to which I have not found the proper answer yet.

My doubt concerns the interpretation of the results of a clustering algorithm which was run on features to which a log-transformation was applied.

Specifically, let's assume we want to run a k-means algorithm on 3 interval variables. Unfortunately, these three interval variables are extremely bad distributed and the k-means gives the worst result we have ever seen. However, let's imagine that by applying a log transformation to each variable, we obtain three incredibly perfect normal distribution.
Then, we run again the k-means and we obtain perfect clusters.

Now, my doubt concerns the interpretation of this cluster obtained by running a k-means on three log-transformed variables: it is not clear whether our interpretation of the clusters obtained should be made on the original variables or it should be made on the log-transformed variables?

Clearly, my example is related to log-transformation but we can talk about z-score or min-max normalization or any other kind of transformation that we apply in order to improve the quality distribution before running the clustering algorithm.

To clarify, what I mean by interpretation is the profiling of the cluster, which means try to describe which are the characteristics common to the individuals belonging to that cluster.

$\endgroup$

1 Answer 1

4
$\begingroup$

Very interesting! What you did to your data is simply a feature mapping/transformation. So how this affects the clustering results?

Clustering is not a clearly defined problem but at least we know something about it: It's about internal similarities (patterns) so these similarities should be maintained through the feature transformation. In your example if you found clusters in transformed space, it shows that you have had clusters in original space as well. You just couldn't see them according to the algorithm you used in that space!

For instance if you use Kernelized versions of algorithms you easily find that what they do is nothing but what you did as transformation. They first use a kernel to map the data into new space and then use the algorithm in that space (of course with a bit of theoretical differences/constraints).

To summarize, no transformation produces a fake pattern in the data. In worst case it vanishes the original pattern and in the best case it reveals the pattern which was not visible originally (which is your case).


I mentioned Fake Pattern above so let me say a bit more on it. I think there is a fundamental issue concerning your question:

You assume that there is a Right clustering that you got after transformation. Actually there is no right clustering!

We do not have fake pattern! If there is a pattern in a feature space, then that is true! i.e. you found an interesting representation of your data. If it does not match with the labels then either the data is very noisy or wrong features have been chosen to represent classes (maybe more reasons. just these two came to my mind now). If there is no label (your case) be sure there is a correlation between features of those cluster members.

$\endgroup$
4
  • $\begingroup$ Amazing! Thank you very much for effectively answer my existential doubt. :) $\endgroup$
    – Seymour
    Dec 12, 2017 at 13:24
  • $\begingroup$ I'm glad it helped :) $\endgroup$ Dec 12, 2017 at 13:25
  • $\begingroup$ can you please share any source interesting to the topic? It could a useful reference in my thesis $\endgroup$
    – Seymour
    Dec 12, 2017 at 13:26
  • $\begingroup$ Having a look at famous paper "An Impossibility Theorem for Clustering" is useful however it's not my favorite paper. I recommend following people like Shai-Ben David and his research. There is no ref explicitly for your question but exploring above mentioned might redirect you to the right place. www-users.cs.umn.edu/~kumar001/dmbook/ch8.pdf looks good as well however i did not go through. $\endgroup$ Dec 12, 2017 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.