1
$\begingroup$

I'd like to know what would you do in this specific and unrealistic case appliying Knn when k = 1, k = 2 and k = 3.

  • Class 1 individuals: [1,1] [2,2] [2,4] [2,5]

  • Class 2 individuals: [3,1] [4,1] [4,2]

  • individual to classify: [2,1]

Plot:

Knn test plot

I don't know if there would be any criteria rather than the generation order to choose the class it belongs.

$\endgroup$
  • $\begingroup$ k=3 seems reasonable (or 4). $\endgroup$ – user2974951 Dec 20 '18 at 11:04
1
$\begingroup$

When k=3 it is a simple case, the 3 nearest examples share the same distance to the new sample. Two of them are from class 1 and a single example is from class 2, so the new sample will be classified as class 1.

When k=2 it is more tricky. Assuming that the 2 features (x,y) share similar importance to the classification (otherwise distances should be calculated with some predetermined weights), you can:

  1. Choose randomly 2 out of the 3 candidates (you are basically acknowledging that for this case, your model doesn't have enough information for classification based on 2-NN).

  2. take all candidates into account (i.e. for cases like this you go to 3-NN).

When k=1, same options as with the k=2 case (you will still need to go up to 3-NN).

In this specific case, because your 3 nearest neighbors are all of the same distance, I would choose to use K-NN. However for a similar case with 4 samples at the same distance, 2 of each class, I would choose randomly (not going to 5-NN).

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.