I am trying to tune the "n_neighbors" for a kNN model andI have the following problem :
Based on the mean cross validation score the optimal kNN model should be the one with 10 neighbors.
On the other hand, when I plot the "scores vs neighbors" graphs I see that there are models whose score distance between the training and the test data is much smaller ( for instance the model with 20 neighbors ).
I am new to ML and this is still very confusing to me.. but should I stick to the 10 neighbors model, or is the 20 neighbors model better ? How do I decide ? Any help is much appreciated.
Here is my code and the graphs :
best_score = 0
neighbors = np.arange(1,31)
all_train_scores = []
all_test_scores = []
for n_neighbors in neighbors :
reg = KNeighborsRegressor(n_neighbors = n_neighbors, metric = 'manhattan')
score = cross_val_score(reg, X_train, y_train, cv = 5)
score = np.mean(score)
if score > best_score :
best_score = score
optimal_choice = {'n_neighbors' : n_neighbors}
reg.fit(X_train, y_train)
train_score = reg.score(X_train, y_train)
test_score = reg.score(X_test, y_test)
all_train_scores = np.append(all_train_scores, train_score)
all_test_scores = np.append(all_test_scores, test_score)