I am a beginner doing my first ML project. I am doing a binary supervised classification on an unbalanced dataset and want to use the ROC curve as a performance metric of my models. I am using Logistic Regression, Support Vector Machine and K Nearest Neighbors as classifiers. For Logistic Regression I understand what the ROC Curve is and how the threshold could be adjusted. When predicting the probabilities for K Nearest Neighbours however I don't understand where the ROC Curve comes in handy, as the threshold can not be varied anyways. So is there a reason to still implement a ROC Curve for K Nearest Neigbors?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
ROC curves (and the AUC metric) are used for evaluating model performance on a classification task. If you use KNN for classifying, then you can evaluate your model on it.
Probability, in the context of KNN can be the number of neighbours that correctly classify an instance (the threshold), divided by the total number of neighbours used (the k parameter).
E.g., you have a KNN model f with k=5, and for data instance x, f(x) = 4 / 5 if 4 of its 5 closest neighbours belong to the same class as x. If for data instance y you have f(y) = 3 / 5, then both x and y would be assigned the same class, but x should rank higher than y because the model is more confident about it (4/5 vs 3/5).
If you can implement your KNN algorithm to work like this, than ROC curve / AUC metric is straightforward to implement.
I think that this is what's implemented in the predict_proba method of the sklearn's KNN classifier.
