# How to appropriately set weights for weighted KNN

I'm currently trying to predict a continuous variable using KNN. Instead of treating each neighbor equally I would like to use the weights to create a weighted average. The weights by themselves are not ideal, as the closer a neighbor the more I would like that neighbor to influence the final results.

This lead me to consider the inverse of each of the distances, but this doesn't handle the case where an instance is the exact same -> with a distance of 0.

Any recommendations on how to properly set the weights of each neighbor relative to their distance? Similar to how the inverse would handle this, but one that allows for 0 values.

A decaying exponential of the form $e^{-\alpha x}$ (where $x$ is the distance from the observation) is convenient in this situation. It has the nice feature that the weight is equal to $1$ when the observation lies exactly at one of your training points and decays to zero as $x\rightarrow \infty$.
What you need to decide is the scaling factor, $\alpha$, which can greatly affect your results, since the decaying exponential is nonlinear. If $\alpha$ is too small or large, then all $k$ points will be weighted nearly equally (assuming none of the $k$ points have a distance very close to zero). One approach is to pick a global value of $\alpha$ that is suitable for your data set. Another approach is to compute a new value of $\alpha$ for each set of $k$ neighbors under consideration, which will guarantee variability in the weights (as long as all $k$ neighbors are not equidistant).