I'm using DBSCAN clustering on a set of documents. The documents' content was converted to TF-IDF matrix, and I'd like to find consistent ways to evaluate the clusters when no added information is given (labels etc.).

  1. Metrics comparing clusters - a score for each cluster - the goal here is to figure out which clusters are 'better' than others (more dense and more unique). Thought of two metrics:

    *Intra-cluster similarity - measure the similarity of documents inside each cluster - I used the cosine distances between documents in the TF-IDF representation

    *Inter-cluster similarity - measure the similarity between clusters (to highlight the more unique clusters) - similarly, planning to use the cosine distances between clusters' kernels

  2. Metrics comparing clustering algorithms - a score for the clustering technique (such as Silhouette score). Will be used for hyper-parameter tuning and choosing an algorithm.

Any ideas?

None of the internal evaluation metrics will work well on text in my experience. Probably because of the curse of dimensionality.

Furthermore, DBSCAN does not cluster everything, but can also produce noise points. Few evaluation methods (and even fewer implementations...) handle it well that noise is not a cluster.

  • Clustering textual files with very similar format, I already tested different algorithms, and so far DBSCAN produced satisfying results. Do you know any metric that can be useful as a rule of thumb? – Adam Dec 13 '17 at 9:39
  • There is DBCV for DBSCAN type clusters. – Anony-Mousse Dec 13 '17 at 19:16

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.