1
$\begingroup$

I have some SVM classifier outputting final classifications for every sample in the test set, something like

1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1

and so on.

The "truth" labels is also something like

1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1

I would like to run that svm with some parameters, and generate points for the roc curve, and calculate auc.

I could do this by myself, but I am sure someone did it before me for cases like this.

Unfortunately, everything I can find is for cases where the classifier returns probabilities, rather than hard estimations, like here or here

I thought this would work, but from sklearn.metrics import plot_roc_curve is not found!

anything online that fits my case?

Thanks

$\endgroup$
5
  • $\begingroup$ The points on a ROC curve are obtained by varying the threshold over a predicted score or probability, so if you only have the final "hard" predictions you can't do a ROC curve. $\endgroup$
    – Erwan
    Dec 7 '19 at 17:38
  • $\begingroup$ @erwan can't i change the parameter manually and plot with some library that knows how to calculate false/true positive? $\endgroup$
    – Gulzar
    Dec 7 '19 at 18:20
  • $\begingroup$ it's not about calculating the TP/FP: from your current output which is a single set of predictions, you will get 1 value for true positive rate and false positive rate, i.e. 1 single point for the curve. you need to have different "series" of predictions, each corresponding to one point, and this is possible if you get the probability instead of just the binary prediction, because you obtain different "series" of predictions by moving the threshold over the probabilities. $\endgroup$
    – Erwan
    Dec 7 '19 at 18:48
  • $\begingroup$ @Erwan I do have control over the C parameter of the LinearSVC. Can't I just choose many values, train it for them, and see all the values that come out? $\endgroup$
    – Gulzar
    Dec 7 '19 at 18:54
  • 1
    $\begingroup$ you could try, but it's unlikely to give you the same kind of variation as varying a threshold so you might end up with just a bunch of points in the same area instead of a ROC progression. If you really want a ROC and can't have probabilities you could train a regression model instead of a classification model: in the training set 0 and 1 would be treated as real numbers, so in the test set you would get predictions mostly between 0 and 1: these predicted values could be used as "probabilities". $\endgroup$
    – Erwan
    Dec 7 '19 at 19:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.