1
$\begingroup$

I have a dataset of 25 instances these instances are divided into 2 classes Green Circles and Blue Squares

data distributed as this graph

enter image description here

I want to predict X's class based on "Likelihood Weighted KNN with k =3"

In normal KNN this is easy

the nearest 3 points are 2 Blue Squares and 1 Green Circle

which means X will be Blue Square

there are more Blue Squares neighbours than Green Circles (2 vs 1)

But What is needed is to find the Likelihood Weighted KNN with k =3

This is my try

In this case we have to calculate the weight (Likelihood) for each instance

Each Green Circle likelihood is $\frac{1}{5} $ , we have 5 Green Circles

While for Blue Squares it is $\frac{1}{20} $ , we have 20 Blue Squares

Therefore the weights around X will be $\frac{1}{5} $ Green Circle, and $\frac{2}{20} $ Blue Squares.

which means $\frac{1}{5} > \frac{2}{20} $

Then X is Green Circle

Well, this is wrong :(

Can someone help me find the

Improve this question
$\endgroup$
1
  • $\begingroup$ Your other post of this question is clearer. I'd suggest removing this one in favor of that one. $\endgroup$
    – Ben Reiniger ♦
    Jul 10, 2019 at 12:13

1 Answer 1

Reset to default
0
$\begingroup$

Green circles and blue squares are samples of two different classes. How does it matter that "X" belongs to which one of either of these "Green circle" or "Blue Square" class?

Likelihood should be done based upon the number of classes and the K value, instead of dataset samples.

For me likelihood weight will be:

Likelihood of being blue square ~ 0.66 i.e. 2/3, whereas likelihood of being green circle is 1/3.

Prepared another example: Where I have 2 Green circle and 16 blue square, whereas K=5 enter image description here

If I follow your approach, likelihood of Green Square is 1/2, Blue square is 4/16 = 1/4, whereas from the image it's clear that "X" belongs to Blue Square class only.

[Update]
Thanks @Ben Reiniger, for correcting me.

I spent more time and what I understood is, it's not always correct to put equal weights(likelihood) across different classes in dataset. My earlier observation was based upon the assumption that all classes have equal weights and I was wrong, even though data is bit skewed.

Consider an another example:

We have a huge dataset of patient reports, where features are created based upon the several tests done so far. Our task is to identify the possibility if a particular person suffering from cancer or not.

In this case, even if a single feature point towards cancer disease, we cann't neglect it and should predict towards rare class, such that patient can go for further analysis.

In such cases weights are not equivalent across different classes. According to me, wights should be defined based upon the use case.

A very well explained its calculation in How does Weighted KNN works?

Improve this answer
$\endgroup$
3
  • $\begingroup$ This seems to miss the point of the likelilhood weighting. (Again, I'd suggest looking at the other posting of the same question.) $\endgroup$
    – Ben Reiniger ♦
    Jul 10, 2019 at 14:31
  • $\begingroup$ Will be good if you share the link as well, thanks. $\endgroup$
    – vipin bansal
    Jul 10, 2019 at 16:40
  • $\begingroup$ so what is the answer? $\endgroup$
    – asmgx
    Jul 10, 2019 at 22:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.