If we don’t specify the distance, how is the n_neighbors calculated?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ Do you mean in a specific software package? // I would expect the default in a software package to be regular Euclidean ($L_2$) distance, and I would be flabbergasted to find that not to be the case in a standard package like sklearn. $\endgroup$– DaveJul 13, 2021 at 2:58
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
This question needs to be more specific. And there might be confusion.
- n_neighbors, is the number of proximity neighbors that the algorithm uses.
- metric, is how you define what is the closes neighbor, by default, is Euclidean.
metric, default=’minkowski’ the distance metric to use for the tree. The default metric is minkowski, and with p=2 is equivalent to the standard Euclidean metric. See the documentation of DistanceMetric for a list of available metrics. If metric is “precomputed”, X is assumed to be a distance matrix and must be square during fit. X may be a sparse graph, in which case only “nonzero” elements may be considered neighbors.
answered Jul 12, 2021 at 7:37
-
$\begingroup$ To add to your great answer, I would also point out that n_neighbors is not a model's parameter, but a hyperparameter, so it is not learn from the data (by the model) but set by the user and "optimized" by CV.So there might be also a confusion on this. $\endgroup$– MultivacJul 12, 2021 at 21:19
-
$\begingroup$ @JulioJesus what will be teh difference bt a parameter and a hyperparameter in this case? $\endgroup$ Jul 12, 2021 at 22:06
-
$\begingroup$ @CarlosMougan Julio is right, what is given to the model is called Hyperparameter as this is given to the model and not calculated by it. The SciKit doc call it parameter in standard API/programming language, but in ML language this is an Hyperparameter. For Knn there is no parameter, as there is in linear regresslion, the line slope found by ML is the parameter calculated by the algo. $\endgroup$– MaloJul 12, 2021 at 22:13
-
2$\begingroup$ I want to clarify that I refer the confusion might be in the question itself, not in Carlos' answer.The difference I highlight is that the question says "...how is the n_neighbors calculated?" and in this case the value of n_neighbors is not calculated by the model but given by the user $\endgroup$– MultivacJul 12, 2021 at 22:17
$\begingroup$
$\endgroup$
n_neighbors is the number of neighbors to take into account. This value is chosen and set by you when you program the KNN model. You can test different KNNs with different values for it, and it is called an hyperparameter. You can use GridSearchCV to test for different values of it.
Then the KNN model will look for the closest n_neighbors (of the new data point) calculated with the chosen metric (when not specified, Euclidian distance is used), and make its prediction based on the n_neighbors found this way.
