0
$\begingroup$

I have a large database of people and I want to show a small number of people who are similar to each person in the database. So if one of the people was Wolfgang Mozart I would want to show Beethoven, Haydn, Schubert etc based upon they are "similar" in that they are roughly contemporary with the same occupation (i.e. Composer). For each person, I have a short occupation which usually contains the nationality (e.g. Austrian composer, American actor, British politician) and dates of birth/death.

Is there a recognised algorithm or technique to do something like this? I don't have a training set as I haven't manually produced any examples of what I want to see, so I don't think machine learning is appropriate.

If I had to write something right now I'd probably just do a string comparison and order so the closest contemporaries (i.e. dates of death) come first.

But is there a better way?

|improve this question
$\endgroup$
  • 1
    $\begingroup$ Clustering is much harder than similarity search (because it uses similarity search). You want similarity search only. $\endgroup$ – Anony-Mousse Nov 8 at 7:05
  • $\begingroup$ Coming up with even just toy-level data will help you clarify the problem. $\endgroup$ – ChrisR Nov 8 at 16:38
1
$\begingroup$

Consider the R package poLCA. Given a set of people with attributes that have categorical values, it can cluster the set into a fixed number of groups. As Anony-Mousse says above, it's not quite the same problem. Similarity would be to find people that are close to the one in the query. Clustering is close to answering that question for everybody.

The name of the package gives some deeper insight into those differences. LCA is for latent class analysis. The idea is that there is some characteristic of the people in the dataset that can be used to group them into clusters, but it's not explicit in the variables you have. It's latent.

|improve this answer
$\endgroup$
0
$\begingroup$

If anything changes and you decide that maybe clustering (unsupervised learning) is a good path to head down there is a great article here that demonstrates different clustering techniques: https://medium.com/datadriveninvestor/unsupervised-learning-with-python-k-means-and-hierarchical-clustering-f36ceeec919c

The only catch is that you might have to modify your data to have features that will work with the model (both training and testing). I believe it only takes integers, so any string values you have you would need to convert them (i.e. for occupation Composer could then be a 3).

A similarity algorithm sounds much easier to produce, but it would need to be manually tweaked more often in production than a model would.

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.