0
$\begingroup$

When applying a nearest neighbors-based method to a data of, for instance, 2000 points, what is the largest number of neighbors that can be considered ? I am using a nearest neighbors method in an unsupervised fashion for anomaly detection.

$\endgroup$
1
$\begingroup$

The predicted value by the nearest neighbor algorithm will actually just take the average value of all points that are close to the new point.

It's up to you to find the number of neighbors that you want to consider. If you have 2000 data points, this are the 2000 'neighbors' of which you will select a number of k neighbors for taking the average and using that average as prediction.

So the maximum number of k is the number of data points in your training set. In your case this is 2000.

$\endgroup$
  • $\begingroup$ Is there any technique that allows determining the well-performing K ? could this be done manually? $\endgroup$ – Melinya Oct 9 '18 at 8:39
  • $\begingroup$ You can use cross-validation $\endgroup$ – ignatius Oct 9 '18 at 8:48
  • $\begingroup$ @ignatius I don't know if it can help as if I split my data, then I don't know what subset size is most representative of my dataset. $\endgroup$ – Melinya Oct 9 '18 at 9:30
  • $\begingroup$ If you want to know whether your k is well performing, you should define what your performance measure is. For this we should understand what problem you are trying to solve. Please give some more detail on this. Cross-validation is often used to tune hyper parameters. And often this is validated on a test data set which is about 30% of the observations of training data. But again this is a very general remark and it may be irrelevant in your case, especially since you talk about unsupervised methods. $\endgroup$ – Joos Korstanje Oct 9 '18 at 12:43
  • $\begingroup$ k-NN is a non parametric method, thus the size of your dataset will have a critical impact on the final performance. You have also to keep in mind the course of dimensionality. What I would do is to configure some experiments: I'd select different sizes of the dataset and for each of them, different number of neighbours. Then, I'd perform cross-validation to have statistical results which can help you evaluate this configurations $\endgroup$ – ignatius Oct 10 '18 at 9:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.