Why do I not get 100% Accuracy with KNN with $K=1$

Question

I am playing with KNN on the Iris Dataset

I expected to get 100% accuracy with $K=1$ since every point should predict itself based on the Voronoi volume around it created by the KNN algorithm.

However using Scikit Learn I do not find this result. Here is my code.

import pandas as pd
import numpy as np

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import confusion_matrix
from sklearn.datasets import load_iris 

iris = load_iris()

X = pd.DataFrame(iris['data']) 

X.columns = ['sepalLength','sepalWidth','petalLength','petalWidth']

featureX = 'sepalLength' 
featureY = 'sepalWidth'

X_2 = X[[featureX,featureY]]

y = iris['target'] 

y = (np.array(y)==2).astype(np.int)

knn = KNeighborsClassifier(n_neighbors=1, p=2) 
knn.fit(X_2, y)  
y_pred = knn.predict(X_2)  
cm = confusion_matrix(y, y_pred)  

print(cm)

I get the following output for the confusion matrix. I see 11 out of 150 samples are incorrectly classified.

[[95  5]

 [ 6 44]]

Why is it not perfect ?

Answer 1

In your training set (X_2,y), there are some samples with the same input features X_2 but different labels y. For example, the 73rd and 147th samples, which are labelled into class 0 and 1, respectively, have the same input values [6.3, 2.5]. There are more samples like this in the dataset. Therefore, you could not construct a perfect classifier for such data.