# K-Nearest Neighbour Classifier Best K Value

I created a KNeighborsClassifier for my dataset adjusting the k hyper-parameter (the number of neighbours) in a for loop. The k value was between 1 and 20. The result was the graph below:

How do I interpret this graph? Which would be my best k value?

Thanks.

There's several ways that you can choose your k value for kNN -

You can use the common formula k = sqrt(n) where n is the number of data points in your training set or you can try choosing k where there is a good balance between computation expense vs noise.

Consider your what fits your problem: Do you care about runtime? The higher the k, the more expensive computationally it is to run.

It looks like around k= 6-10 you get some diminishing returns - you could set it there to get a good balance between noise and computation cost, but ultimately it is a very arbitrary selection, so pick what suits your use case best.

That is called an Elbow-Curve! You need to look for the lowest train accuracy, or highest test accuracy, where the curve doesn't bend much more (y-axis), for a given increase in K (x-axis). In your case, it's around 7, but you could make an argument for 10, I suppose. As you increase your 'K' the accuracy improves, but only incrementally, and the cost (total compute resources, overall time to finish, etc.) will increase quite significantly.

Here is another example of a similar Elbow-Curve.

In this example, a good K would be around 7.

Here is some sample code for you to test, and further illustrate the point.

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
#%matplotlib inline

df.info()

df.describe()

l=list(df.columns)
l[0:len(l)-2]

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()

scaler.fit(df.drop('TARGET CLASS',axis=1))
scaled_features = scaler.transform(df.drop('TARGET CLASS',axis=1))

df_feat = pd.DataFrame(scaled_features,columns=df.columns[:-1])

from sklearn.model_selection import train_test_split
X = df_feat
y = df['TARGET CLASS']
X_train, X_test, y_train, y_test = train_test_split(scaled_features,df['TARGET CLASS'],
test_size=0.50, random_state=101)

from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=1)
knn.fit(X_train,y_train)

KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
metric_params=None, n_jobs=1, n_neighbors=1, p=2,
weights='uniform')

pred = knn.predict(X_test)

from sklearn.metrics import classification_report,confusion_matrix
conf_mat=confusion_matrix(y_test,pred)
print(conf_mat)


Result:

              precision    recall  f1-score   support

0       0.88      0.90      0.89       250
1       0.90      0.87      0.89       250

accuracy                           0.89       500
macro avg       0.89      0.89      0.89       500
weighted avg       0.89      0.89      0.89       500


Continue:

print(classification_report(y_test,pred))

print("Misclassification error rate:",round(np.mean(pred!=y_test),3))

error_rate = []
# Will take some time
for i in range(1,60):

knn = KNeighborsClassifier(n_neighbors=i)
knn.fit(X_train,y_train)
pred_i = knn.predict(X_test)
error_rate.append(np.mean(pred_i != y_test))

plt.figure(figsize=(10,6))
plt.plot(range(1,60),error_rate,color='blue', linestyle='dashed', marker='o',
markerfacecolor='red', markersize=8)
plt.title('Error Rate vs. K Value', fontsize=20)
plt.xlabel('K',fontsize=15)
plt.ylabel('Error (misclassification) Rate',fontsize=15)


# sample data avalable here:
# https://www.kaggle.com/shubh247/simple-way-handle-classified-data-using-knn/data