0
$\begingroup$

I am learning about KNN and ML in general. I know that KNN usually uses second order Minkowski distance (Eucledian Distance), but I assume it cal also use other orders. But what is the benefit to choosing a higher order with respect to the general performance of the model? Is it faster? More accurate? Does the level of noise influence my decision for the order? How do I know that because of this and this condition, I have to use a higher order Minkowski distance?

$\endgroup$
0
$\begingroup$

Unfortunately, there is no rule (either general or rule of thumb) for using a given distance rather than another. Minkowski distance of n-th order could work greatly for some task and some datasets, and extremely bad for others.

The reason Euclidean distance is used as default is due to its simplicity and the fact that there is no reason to consider more complex formulae as better a priori.

I suggest you to try multiple measures of distance (or multiple Minkowski orders) and check which works best for your current task. Please consider the risk of overfitting. I would run models more than once, for example using techniques for keeping overfitting under control, such as k-fold Cross Validation, and/or triple train-validation-test splits.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the explanation! I have one more question. Generally speaking, is the order of the distance in any way correlated with overfitting the data? $\endgroup$ – P_Andre Jan 29 at 17:24
  • $\begingroup$ I can't think of any mathematical reason for it. I might be wrong, but personally I don't think so. $\endgroup$ – Leevo Jan 29 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.