I have a list of words (names actually) on which I would like to apply some entity resolution. My first guess was to create clusters of similar names so I could extract a representative entity from multiple name shapes.
I need to specify that I have no labelled data, and I am not working on a document analysis (this is different from Improve results of a clustering for example), only a raw list.
To do so, and based on what I could read, I attempted two approaches :
apply n-gram transformation on my names and use k-means clustering
apply n-gram transformation, compute a similarity matrix (cosine distance) and use it for affinity propagation
Both approaches give me interesting results, yet I can't understand some of the results. For example, I get the following clusters :
Geronese, Varonese, Veronefe, Veronese, Veronesse, ...
Cameroni, Veronèse, Veronèse P., Veronése, Veronêse
Why do I get two different clusters for shapes that look so similar (except for Cameroni
which I don't know why it is in that cluster) ? Is this a problem in the k-means algorithm tuning ?
Also, I tried using the silhouette metrics to find the optimum number of clusters but I get the exact same value no matter what is the number of clusters (0.315 for what it's worth).
As for the affinity propagation approach, I get a lower silhouette score for my clusters, and I get some similar effects, like having this kind of cluster :
Birttetti, Laruette, Laruelle, Larvette, Laurette, ...
Any ideas how I could improve my results (if this is possible) ? Or maybe any idea for a better approach than mine ?