1
$\begingroup$

I'm working on topic modeling and I have generated clusters with two different methods.

How can I evaluate which method performs better than the other?

enter image description here

$\endgroup$
1
  • $\begingroup$ Hi, out of curiosity, could you add the methods you've used? $\endgroup$ Commented Jan 7 at 6:58

1 Answer 1

1
$\begingroup$

Evaluating unsupervised learning methods is always an interesting question. There are typically two main ways to evaluate clusters.

Explicit evaluation

Qualitative analysis

First of all, you should always manually inspect the results to make sure they make sense to you. In practice, this is hard to beat and probably what you would use as a tie-breaker between methods who perform similarly on hard metrics.

How many clusters?

Different methods might yield varying numbers of clusters. Methods like K-Means force you to set the number of clusters in advance, but others like Mean Shift do not.

If you already have some intuition about how many clusters is desirable, it can already be used as a way to discriminate against the results of certain methods (i.e. yielded too many or too few clusters)

Is it really unsupervised?

Even though the model might be learned in an unsupervised way*, it doesn't mean that the evaluation must also be! A good way to validate clustering results is to have pairs of data points that you know should or should not end up in the same cluster. You can then use regular metrics like accuracy to measure how well your clustering results satisfy your preferences/constraints.

* some clustering methods are semi-supervised and can use preferences as input

Implicit evaluation

Obviously, there should always be some hard metrics you can compute to compare different clustering results.

Stability

Running the clustering method on different subsets of the data or with a different seed (i.e. for something like K-Means) can teach you about how stable your clustering technique is. An "ideal" clustering technique would always cluster the same data points together.

Cohesion & Separation

The data points within the same cluster should be close to each other (cohesion) and far from data points in other clusters (separation). How you measure distance and the shape of your clusters obviously impact this metric greatly. The silhouette score for instance works best with convex clusters.

This link provides a few evaluation metrics and describes them quite well:

  • Rand index
  • Mutual information
  • V-measure
  • Fowlkes-Mallows
  • Silhouette coefficient/score
  • Calinski-Harabasz
  • Davies-Bouldin
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.