# How to interpret silouette coefficient?

I'm trying to determine number of clusters for k-means using sklearn.metrics.silhouette_score. I have computed it for range(2,50) clusters. How to interpret this? What number of clusters should I choose?

## 2 Answers

They are all bad. A good Silhouette would be 0.7

Try other clustering algorithms instead.

• I'm clustering text data. So, I can compute the most popular word in each cluster. When I see at top_n words, I can conclude that algorishm works well. But I don't know what k is the best number of clusters. I don't understand why this metric shows so bad result. Oct 8, 2016 at 22:49
• Pick 10 random documents. Assign every document to the most similar of them. The top_n words will usually still look good... Oct 9, 2016 at 0:07
• But I can look at documents and almost all documents corespond to this words. Oct 9, 2016 at 9:43
• Yes, but so would they in the random example I gave. A more relevant evaluation is word coherence. see "Reading Tea Leaves: How Humans Interpret Topic Models", papers.nips.cc/paper/… Oct 9, 2016 at 10:14
• Sorry, I looked at the words that correspond to the center of cluster. Oct 9, 2016 at 10:38

Silhouette measures BOTH the separation between clusters AND cohesion in respective clusters.

Intuitively speaking, it is the difference between separation B (average distance between each point and all points of its nearest cluster) and cohesion A (average distance between each point and all other points in its cluster) divided by max(A,B).

It is a value between -1 and 1, the higher the better (negative value means that the point is more closer to the nearest cluster than to its own, which is quite a problem).