So I recently came along kNN k nearest neighbour. When looking at its disadvantages, most of the literature mentions it is costly, lazy, requires full training data plus depends on the value of k and has the issue of dimensionality because of the distance. Other than that I have following hypothesis.
1- It ignores the fact that dimensions can be inter related and instead assumes they are independent (as we are just calculating distance) 2- Has the issue of normalization of data... if the data is not normalized distance can be biased towards a specific dimension
I will like to have a comprehensive analysis on the disadvantages of kNN apart from those mentioned above and if they are wrong then why.