1
$\begingroup$

I am doing a project on finding famous people who are similar to each other.

For this, I am extracting a bunch of features and applying a distance function on them to evaluate who is closer to whom.

Is there a way to test this out quantitatively? Take the example of actors for easy reference.

Say, one model predicts Robert Downey Jr. to be most similar to Christian Bale. While after applying more weights to the 'heights' feature, George Clooney is found to be most similar to Christian Bale.

Now how do I quantitatively assert that one is better than the other?

One might say that Robert Downey is a better result because both Christian Bale and Robert Downey have starred in cult superhero movies (albeit different ones), while someone else might say that since George Clooney and Christian Bale were both batmans at one point, George Clooney is the better result.

Is there a way to test this other than intuition?

$\endgroup$
2
$\begingroup$

Not unless you have an idea of what makes two people similar to each other. By choosing a distance function you are defining what it means for two people to be similar to one another. The question of whether one metric is quantitatively better than another may not be answerable without a defined objective.

For example, compare the Manhattan distance (L1 norm) to the euclidean distance (L2 norm). The former may be more effective for a pedestrian trying to navigate a city on foot, whereas the latter may be more effective for an athlete running across a sports field.

Now how do I quantitatively assert that one is better than the other?

You would need some examples which embody what it means for two people to be similar (or not) to one another. This would have to be generated by a person or group of people.

Rather than generate a "similarity value" directly between two people, one could instead generate a ranked list of people which are most similar to a given person of interest: For person A, the most similar other person is B, the second most similar person is C, etc. Better yet, presented with a person of interest and a list of other people someone would just have to rank each person in the list by their similarity to the person of interest.

If more than one person generate ranks of a given list of people for a given person of interest, you could agglomerate the results by summing the ranks for each person and reordering the list by summed rank.

If you have some data in this form, then you may evaluate your similarity model by how well it recreates the ranked lists. Some possible metrics for evaluation are:

  1. Recall (at rank n): What fraction of the top n people are returned in the model's top n results?
  2. Precision (at rank n): What fraction of the model's top n results are in the true top n results?

Look here for other methods of evaluating ranking methods.

I hope this helps. Let me know if I'm not understanding your question.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.