# Can k=1 be a good choice for K neighbors classification?

Running sklearn.KNeighborsClassifier() on Kaggle's Leaf Classification sample (set of 99 species, 10 specimen each), with defaults kNN parameters and a grid search optimization (using sklearn.model_selection.GridSearchCV), I find the best accuracy is achieved for k=1.

Although this is a technically valid choice of k, I feel using a single neighbor means the model wasn't able to find a strong relationship between neighbors.

Am I right, or is k=1 sometimes a good choice, and for which kind of problems ?

Note that using neg_log_loss as scoring parameter leads to an optimal number of k=5 neighbors. I am still struggling to understand how this is an improvement since the accuracy is smaller then.

Update : I was able to find more information using the "1NN" and "1-NN" keywords, which suggest this is indeed a good choice in some situations.

Having k = 1 is not inherently unreasonable. This just means that all new observations will be predicted to be the class label of the first closest neighbour you have in your training set. Predicted class probabilities should just be 1 or 0 since the the estimator will take on the form 1/1 or 0/1. This also makes it clearer as to why log loss will probably not favor a k = 1 model; the predicted probabilities will be far too extreme in the cases where the model favors the wrong class label. Consider the case where the the true label is 0, i.e. $$y_i = 0$$. If the closest neighbour in your training set is of class label 1, then your model will incorrectly predict $$Pr[Y = 1|X] = 1.$$ Hence, $$(1-y_i) * ln(1-Pr[Y = 1|X])$$ will be infinity. On the other hand, if the true label is 1, i.e., $$y_i = 1$$, and the closest neighbour to this observation ends up being of label 0, now $$Pr[Y = 1|X] = 0$$ and we have $$y_i * ln(Pr[Y = 1|X])$$ infinity as well. Thus, log loss will either be 0 (where your model predicts correctly) or -infinity for all of your observations; clearly very extreme.