# K-Nearest neighbor in transformed space

When googling "weighted KNN", the results appear to be focused on weighting the nearest neighbor values after those neighbors have been determined. I'm looking for something that assigns a level of importance to various dimensions that could possibly change the neighbors that are considered nearest with the goal of maximizing the accuracy of predictions made using the resulting model.

For example, if I have a new observation defined as [1, 2], #1 and #2 below would be considered the nearest neighbors assuming K=2 under a normal KNN (as I understand it):

1. [1.5, 2.5] d=0.707107
2. [1.8, 1.5] d=0.943398
3. [4.0, 2.2] d=3.006659

However, if the dimensions were weighted by importance, such as [0.1, 1], the items above would have respective distances of 0.502494, 0.50636, 0.360555 based on sqrt((d1*w1)^2 + (d2*w2)^2), which would make #3 the overall nearest.

Given the goal of maximizing prediction accuracy, I'm wondering what methods are available to accomplish this? I'd like to potentially use this for both classification and regression - are there methods other than KNN that I should consider?

## 1 Answer

You can modify the matrix used to compute the distances between points. By default, KNN with quantitative features is used with Euclidean distances, this is, the identity matrix in the equation (x-y)' I (x-y) (it returns the squared Euclidean distance between point X and Y).

Instead of the identity matrix, you can use other positive definite matrices and compute Mahalanobis distances.

Other modifications you can do is manipulate each feature. This also affects the output of KNN, aside of standardizing, you can try non linear modifications of the features, reducing kurtosis or skewness, for example.

• Is there a structured way to do this that will maximize prediction accuracy? – SuperCodeBrah Apr 26 at 17:34
• Not sure in your case, but I suggest you can try Discriminant Adaptive Nearest Neighbor Classification (DANN). – javierazcoiti Apr 30 at 16:55