values passed to user-defined distance function by KNeighborsClassifier is wrong

Question

I have a data-set in which all features are binary and the class of each data-point is also binary. I am trying to use KNearestClassifier with a user-defined distance function as follows:

KNN = KNeighborsClassifier(n_neighbors=3,
                           algorithm='ball_tree',
                           metric='pyfunc',
                           metric_params={"func": lev_metric})
x_train, x_test, y_train, y_test = train_test_split(df_sum,
                                                    y,
                                                    test_size=0.1,
                                                    random_state=0)
KNN.fit(x_train, y_train)

and my custom metric function is as follows:

def lev_metric(a, b):
    print(a)
    print(b)
    return levenshtein(a, b)

the metric function expects two ndarrays of binary values of 0s and 1s. When knn.fit calls the metric function, "b" looks as expected (e.g. [0 1 1 0 0 1 0 1...) but "a" looks like gibberish and is an ndarray with real valued elements between 0 and 1, for example :

[0.32222222 0.42222222 0.34444444 0.47777778 0.41111111 0.38888889
 0.4        0.31111111 0.35555556 0.35555556 0.42222222 0.46666667
 0.36666667 0.32222222 0.41111111 0.32222222 0.36666667 0.35555556
 0.41111111 0.33333333 0.4        0.42222222 0.3        0.37777778
 0.38888889 0.48888889 0.41111111 0.43333333 0.34444444 0.35555556
 0.43333333 0.38888889 0.43333333 0.32222222 0.47777778 0.34444444...

what am I missing? I have also checked that the "x_train" is correct. Also, isn't knn an instance based learner? why is it calling the distance function in fit anyways? is it not supposed to just memorise the training examples? Thanks.

Answer 1

Neighbors-based methods are known as non-generalizing machine learning methods, since they simply “remember” all of its training data (possibly transformed into a fast indexing structure such as a Ball Tree or KD Tree).

(emphasis mine. Source: https://scikit-learn.org/stable/modules/neighbors.html#nearest-neighbors )

You are using algorithm='ball_tree', which roughly speaking uses Euclidean balls to cluster the points, making use of the ball distances for bounds on the distances between actual datapoints (wiki). So, I suspect the "a" value you're seeing is actually one of the balls' centers.

For the Levenshtein distance, you probably should use the 'brute' algorithm.