I am reading notes on using weights for KNN and I came across an example that I don't really understand.

Suppose we have K = 7 and we obtain the following:

Decision set = {A, A, A, A, B, B, B}

If this was the standard KNN algorithm we would pick A, however the notes give an example of using weights:

By class distribution (weight inversely proportional to class frequency)

class A: 95 %, class B 5 %

This results in a class of B.

I can't seem to figure out the math that was left out to obtain B as the answer.


1 Answer 1


We can view nearest neighbor as a voting process where we consult our $k$ nearest neighbor.

We give the $i$-th data point a voting weight $w_i$.

In your example, each data point in class $A$ has weight $\frac1{0.95}$ and each data point in class $B$ has weight $\frac1{0.05}$. There are $4$ votes from class $A$ and $3$ votes from class $B$. We give class $A$ a score of $\frac{4}{0.95}\approx 4.21$ and class $B$ a score of $\frac{3}{0.05}=60$. Class $B$ has a higher score, hence we assign it to class $B$.

  • $\begingroup$ This makes much more sense now, the percentage 95% and 5% is the class frequency, I thought it was the weights. $\endgroup$
    – Belphegor
    Dec 10, 2018 at 2:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.