For my nlp problem I'm using a combination of TFIDF and KMeans from the sklearn package. The tfidf gets the vectors and then I use Kmeans to cluster the texts based on the vectors. I have a few parameters of the TFIDF that I play around with like the n_gram, the input features, and the stop_words. The problem is how do I evaluate this model? My guess is I don't have to evaluate the KMeans model since its role is just to calculate the distances between points and that I just have to focus on the TFIDF model and parameters I end up using. Is this correct? If it is, how do I evaluate this model apart from manually shifting through the clusters to see if items are grouped correctly?

Edit: I forgot to mention: I don't have a target label. I'm basically grouping texts that look similar in clusters.


1 Answer 1


To evaluate a model based on K-means clustering and TF-IDF, you can use a variety of metrics.

About TF-IDF, you can use metrics from sklearn:

from sklearn import metrics

actual = predictionDF.select('label').toPandas()
predicted = predictionDF.select('prediction').toPandas()
print('accuracy score: {}%'.format(round(metrics.accuracy_score(actual,         predicted),3)*100))

One common metric is the silhouette coefficient, which measures how well each sample has been assigned to its cluster. This coefficient ranges from -1 to 1, with a high value indicating that the sample is well-matched to its cluster and a low value indicating that it is poorly matched.


You can also use other metrics such as the Calinski-Harabasz index or the Davies-Bouldin index, which measure the compactness and separation of the clusters. It's important to choose the right evaluation metric for your particular task and data.

For another unsupervised clustering, you can apply UMAP and TF-IDF. UMAP is a non-linear algorithm for dimensional reduction that works better than linear ones:


In all cases, you need some target information, at least for a random sample (test data). Otherwise, by which basis could you evaluate if the results are correct or not?

See also:



  • $\begingroup$ Thanks for your answer! Sorry I forgot to mention: I don't have a target label. I'm basically grouping texts that look similar in clusters. How do I handle this scenario since there is no target label? $\endgroup$
    – james pow
    Dec 7, 2022 at 20:20
  • $\begingroup$ You're welcome. I've added another solution that might be valid. $\endgroup$ Dec 7, 2022 at 21:26
  • $\begingroup$ I have updated the answer about target data. In fact, you need some to have an evaluation. $\endgroup$ Dec 8, 2022 at 5:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.