1
$\begingroup$

I am reading about KNN

enter image description here

enter image description here

So I made another example to make things clearer

In this example (Image attached)

enter image description here

You can see there are in total 5 Greed Circles and 20 Blue Squares

by standard KNN (k=3) , X should be a Blue Square

This is obvious 2 Blue Squares vs 1 Green Circle.

But in weighted KNN things are difference

In this case we have to calculate the weight (Likelihood) for each instance

Each Green Circle likelihood is $\frac{1}{5} $ , we have 5 Green Circles

While for Blue Squares it is $\frac{1}{20} $ , we have 20 Blue Squares

Therefore the weights around X will be $\frac{1}{5} $ Green Circle, and $\frac{2}{20} $ Blue Squares.

which means $\frac{1}{5} > \frac{2}{20} $

Then X is Green Circle

But if try to think of it logically then there are more Blue Squares than Green Circles which means X more likely to be Blue Square than Green Circle.

My question is :

Am I doing anything wrong here? Can someone explain why the equation is showing Green Circle while logic says Blue Square?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Logically speaking, I think "X is Green Circle" is a reasonable conclusion. I find the idea in the paper in your question is quite similar to this paper: KRNN: k Rare-class Nearest Neighbour Classification.

Intuitively, for example, if a new data point is close to one rare class' point and one common class' point, it is more likely to belong to the rare class.

There's no conflict here because a data point will be much more likely to be close to a common class' point. However, once it's already close to a rare class's data point, it's more likely to belong to this class.

That said, I didn't check your calculation, I just don't think the "X is Green Circle" conclusion is illogical for this algorithm.

[Update]

After reconsidering this problem, I think Weighted kNN want to put an emphasis on the rare class data point because that is the class of interest (e.g. Anomaly detection).

It's possible that Accuary is not the metrics here but a Weighted Accuracy metrics that penalize a misclassified rare class data point harder so that we can detect more rare class data point.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Thanks but sorry i can not understand it, how come the test point should belong to the rare class?? assume midwife career gender classes class F for females with 99% and class M for males with 1%. if there is an unknown member that has the characteristic of F and M. it is more likely to be a female isn't it? why would it be a male $\endgroup$
    – asmgx
    Jul 10, 2019 at 22:34
  • 1
    $\begingroup$ It's more likely to be female, that's true. And if your goal is to obtain the highest accuracy, it makes perfect sense to predict that this unknown point belongs to the female class. However, consider another problem: Fraud detection. Normally, we would like to detect as many fraud transactions as possible, given that we don't make too many mistake (false alarm). In this case, it is better to flag a transaction that's quite similar to a fraud transaction, even those it's still closer to a normal transaction, so that we can investigate it more carefully. $\endgroup$
    – TQA
    Jul 11, 2019 at 9:58
  • $\begingroup$ But this is not my question about false positive. my question about weighted KNN and likelihood. I want to know the way weighted KNN gets calculated. $\endgroup$
    – asmgx
    Jul 11, 2019 at 11:39
  • 1
    $\begingroup$ I think it's answered here: datascience.stackexchange.com/questions/42376/…. About your question: No, logic doesn't say Blue Square. If you use normal kNN, then logic does say Blue Square. If you use weighted kNN, the weight for Green Square is now 4 time bigger than Blue Square, so even when X is close to 1 Green Square and 2 Blue Square, after applying the weight, it's more like 4 unit of green square and 2 unit of blue square, so logically, it should be green. $\endgroup$
    – TQA
    Jul 11, 2019 at 11:53
  • $\begingroup$ in this example he does not mention sample size, how the likelihood of A and B were calculated. $\endgroup$
    – asmgx
    Jul 11, 2019 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.