1
$\begingroup$

I provide an offline library of Music to my users. My goal is to understand what my users are looking for, which means translate raw user searches to: Music Artists, Songs, Albums and then add music to the company library.

What are the suggested clustering algorithms to group common short sentences into a single entity. Example:

  • Taylor Swift Shake it Off
  • Taylor Swift Shaek it Off
  • Shake it off
  • Twylor Swift Shake it Off

I tried this example and works fine for a specific number of clusters (K) where K < N | K <= N. But since searches are unpredictable need to find a way to automate the number of clusters: my goal is to cluster 2 or more similar items and let alone single searches in independent clusters, example:

Cluster 1:

  • Taylor Swift Shake it Off
  • Taylor Swift Shaek it Off
  • Shake it off
  • Twylor Swift Shake it Off

Cluster 2

  • Avicci
  • Avicii Ibiza
  • Avicci Electro House

Cluster 3

  • Juan Gabriel

Cluster 4

  • Adele

Cluster 5

  • Britney Spears
  • Britney Spears VMA 2016
$\endgroup$
3
$\begingroup$

I would suggest hierarchical clustering. It's unsupervised, and you don't need to predefine number of clusters. How it works (for the bottom up version) is each sentence (or object) is initialized as its own cluster. At each iteration of the algorithm, the two clusters of the smallest intra-cluster distance are joined, all the way until there's a 'root' where everything is one cluster. The result of this is a big dendrogram, and you can cut the dendrogram at whatever point you want to define clusters. Or you can just inspect it. Here's a nice numerical example of the alg in action.

It should detect the same types of clusters you're already finding, and you wouldn't need to redefine a distance metric. And, I anticipate you'll get the same amount of class separation you seem to already be getting; in fact, the nice thing about the dendrogram from hierarchical clustering is that it illustrates class separation nicely. it's 'hclust' in R, not sure for Python.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.