1
$\begingroup$

I'm working on classification problem and decided to use KNN classifier for the problem.

so if k=131 gave me auc of 0.689 and k=71 gave me auc of 0.682 what should be my ideal k?

Does choosing higher k means more usage of computational resource? if that's the case can I go with k=71. (or) should I always use K with maximum score no matter what?

Improve this question
$\endgroup$
2
  • $\begingroup$ So, are you calculating auc using cross-validation? $\endgroup$
    – pythinker
    Commented Apr 8, 2019 at 19:16
  • $\begingroup$ @pythinker yes.. $\endgroup$
    – Anjith
    Commented Apr 8, 2019 at 19:26

2 Answers 2

Reset to default
1
$\begingroup$

Because knn is a non-parametric method, computational costs of choosing k, highly depends on the size of training data. If the size of training data is small, you can freely choose the k for which the best auc for validation dataset is achieved. In the case where you have a large training dataset, choosing large k can lead to huge computational complexity which is reflected in slow prediction for test data.

Improve this answer
$\endgroup$
2
  • $\begingroup$ does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values? $\endgroup$
    – Anjith
    Commented Apr 8, 2019 at 19:44
  • 1
    $\begingroup$ Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough. $\endgroup$
    – pythinker
    Commented Apr 8, 2019 at 20:02
2
$\begingroup$

I was taught the best way is to find the error for each k then plot them and look for the "elbow" on the plot.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So I used go with k=131 $\endgroup$
    – Anjith
    Commented Apr 8, 2019 at 18:46
  • $\begingroup$ It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out. $\endgroup$
    – Stephen Ewing
    Commented Apr 8, 2019 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.