In many places, I have seen it only mentioned that predict the label of the query point as the label with more than half of the labels of it's K nearest neighbours. However, I don't see it mentioned that the labels should be 50/50. Is it not required? Or is it bad to use KNN for such data where labels are very much mixed in the space and not clustered in particular regions. Is there any underlying assumptions of this model that I'm missing?

  • 1
    $\begingroup$ Why do you think the labels should be balanced? $\endgroup$
    – Dave
    Commented Dec 4, 2021 at 6:27
  • $\begingroup$ @Dave, I think for an imbalanced dataset, say 90/10 ratio, more often than not, you'll end up predicting class with 90% unless the 10% data is clustered in a particular region in space which was the second part of the question. Also it could be that these regions of 10% classes could have numbers less than K/2. $\endgroup$
    – Hithesh Kk
    Commented Dec 4, 2021 at 6:32
  • $\begingroup$ What’s wrong with predicting the dominant class more often? $\endgroup$
    – Dave
    Commented Dec 4, 2021 at 7:25
  • $\begingroup$ @Dave, after more thought and your input, I think the choice of K takes care of the problems I mentioned like size of minority being very smaller than K. Thank you. $\endgroup$
    – Hithesh Kk
    Commented Dec 4, 2021 at 10:00


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.