So I recently came along kNN k nearest neighbour. When looking at its disadvantages, most of the literature mentions it is costly, lazy, requires full training data plus depends on the value of k and has the issue of dimensionality because of the distance. Other than that I have following hypothesis.

1- It ignores the fact that dimensions can be inter related and instead assumes they are independent (as we are just calculating distance) 2- Has the issue of normalization of data... if the data is not normalized distance can be biased towards a specific dimension

I will like to have a comprehensive analysis on the disadvantages of kNN apart from those mentioned above and if they are wrong then why.

  1. It doesn't handle categorical variables very well
  2. It doesn't handle 'soft' boundaries - i.e. areas where some cases appear on either side of a boundary. See also mathbabe here: https://mathbabe.org/?s=nearest+neighbor - for extended criticism.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.