9
$\begingroup$

Most discussions of KNN mention Euclidean,Manhattan and Hamming distances, but they dont mention cosine similarity metric. Is there a reason for this?

$\endgroup$
  • 2
    $\begingroup$ This problem is called "maximum inner product search". There are efficient solutions. $\endgroup$ – Emre Jan 9 '18 at 17:34
10
$\begingroup$

Short answer: Cosine distance is not the overall best performing distance metric out there


Although similarity measures are often expressed using a distance metric, it is in fact a more flexible measure as it is not required to be symmetric or fulfill the triangle inequality. Nevertheless, it is very common to use a proper distance metric like the Euclidian or Manhattan distance when applying nearest neighbour methods due to their proven performance on real world datasets. They will therefore be often mentioned in discussions of KNN.

You might find this review from 2017 informative, it attempts to answer the question "which distance measures to be used for the KNN classifier among a large number of distance and similarity measures?" They also consider inner-product metrics like the cosine distance.

In short, they conclude that (no surprise) no optimal distance metric can be used for all types of datasets, as the results show that each dataset favors a specific distance metric, and this result complies with the no-free-lunch theorem. It is clear that, among the metrics tested, the cosine distance isn't the overall best performing metric and even performs among the worst (lowest precision) in most noise levels. It does however outperform other tested distances in 3/28 datasets.

So can I use cosine similarity as a distance metric in a KNN algorithm? Yes, and for some datasets, like Iris, it should even yield better performance (p.30) compared as to Euclidian.

| improve this answer | |
$\endgroup$
1
$\begingroup$

as Lejafar mentioned cosine violates triangle inequality however maybe this repo will help you

| improve this answer | |
$\endgroup$
0
$\begingroup$

If there does exist a reason it probably has to do with the fact the Cosine distance is not a proper distance metric. Nevertheless, it's still a useful little thing.

| improve this answer | |
$\endgroup$
  • $\begingroup$ 'Cosine distance is not a proper distance metric' -- Why? $\endgroup$ – Dawny33 Jan 9 '18 at 16:35
  • 1
    $\begingroup$ It fails the triange inequality which is a requirement for a proper distance metric. $\endgroup$ – Tophat Jan 9 '18 at 16:37
  • $\begingroup$ Interesting. I understand that it fails the triangle inequality. But, that that it is a requirement for a proper distance metric, is a new info to me. Thanks :) $\endgroup$ – Dawny33 Jan 9 '18 at 16:41
  • $\begingroup$ View definition of a metric here by en.wikipedia.org/wiki/Metric_(mathematics) $\endgroup$ – Tophat Jan 9 '18 at 16:49
0
$\begingroup$

Although cosine similarity is not a proper distance metric as it fails the triangle inequality, it can be useful in KNN.

However, be wary that the cosine similarity in greatest when the angle is the same; cos(0º) = 1, cos(90º) = 0. Therefore, you may want to use sine or choose the neighbours with the greatest cosine similarity as the closest.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.