I am currently reviewing some concepts related to Machine Learning, and I started to wonder about the hyperparameter selection of K-NN classifier.
Suppose you need to solve a classification task with a number of classes equal to M: I was thinking that the best choice for the parameter K of our classifier should be for K > M.
In this way, we are avoiding all the pathological cases in which a sample may be in the middle of all the M classes and then have a tie. For instance, consider the following example in which we have M=3 and each geometrical shape represents a class:
Assume that K<=M: for sure you will have a tie for a sample in the middle of samples 1, 2 and 3. This tie could be avoided if K > M.
Clearly this is just a toy example, but I think it is sufficient to illustrate my thoughts. I have tried to look for an answer but I wasn't able to find any resource mentioning this, am I wrong in some way or this reasoning may be sound?