When using KNN, how do I know which order of the Minkowski distance to use?

Question

I am learning about KNN and ML in general. I know that KNN usually uses second order Minkowski distance (Eucledian Distance), but I assume it cal also use other orders. But what is the benefit to choosing a higher order with respect to the general performance of the model? Is it faster? More accurate? Does the level of noise influence my decision for the order? How do I know that because of this and this condition, I have to use a higher order Minkowski distance?

Answer 1

Unfortunately, there is no rule (either general or rule of thumb) for using a given distance rather than another. Minkowski distance of n-th order could work greatly for some task and some datasets, and extremely bad for others.

The reason Euclidean distance is used as default is due to its simplicity and the fact that there is no reason to consider more complex formulae as better a priori.

I suggest you to try multiple measures of distance (or multiple Minkowski orders) and check which works best for your current task. Please consider the risk of overfitting. I would run models more than once, for example using techniques for keeping overfitting under control, such as k-fold Cross Validation, and/or triple train-validation-test splits.