I'm using DBSCAN clustering on a set of documents. The documents' content was converted to TF-IDF matrix, and I'd like to find consistent ways to evaluate the clusters when no added information is given (labels etc.).
Metrics comparing clusters - a score for each cluster - the goal here is to figure out which clusters are 'better' than others (more dense and more unique). Thought of two metrics:
*Intra-cluster similarity - measure the similarity of documents inside each cluster - I used the cosine distances between documents in the TF-IDF representation
*Inter-cluster similarity - measure the similarity between clusters (to highlight the more unique clusters) - similarly, planning to use the cosine distances between clusters' kernels
Metrics comparing clustering algorithms - a score for the clustering technique (such as Silhouette score). Will be used for hyper-parameter tuning and choosing an algorithm.
Any ideas?