1
$\begingroup$

Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to something like an ROC curve. I'm interested in doing so because it's important for my project (image classification) to be able to adjust the classification thresholds.

$\endgroup$
9
  • 2
    $\begingroup$ Are you sure you mean to use relu in the output layer? You do not need a bounded output to be able to do a ROC curve (though you might use a different approach to the code if you don’t have probability outputs), though relu would not be the standard choice for the output layer activation function. $\endgroup$ – Dave Nov 24 '20 at 3:20
  • $\begingroup$ You can use sigmoid activation, and then apply the required threshold. $\endgroup$ – Ashwin Geet D'Sa Nov 24 '20 at 12:29
  • $\begingroup$ @AshwinGeetD’Sa Apply sigmoid when, instead of or after the relu? I still think the OP doesn’t mean to use relu in the output layer, though. $\endgroup$ – Dave Nov 24 '20 at 12:34
  • $\begingroup$ @Dave, I meant instead of reLu. The OP mentions "convert the outputs from a ReLU", by this I assume, that OP wants to use ReLU instead of softmax. $\endgroup$ – Ashwin Geet D'Sa Nov 24 '20 at 12:37
  • 1
    $\begingroup$ @AshwinGeetD’Sa Okay, so we’re in agreement that this is unorthodox and might be a mistake. // Mistake or not, I hope to get some time to be able to post an answer in the coming days. You don’t need probability outputs to do ROC (though particular software implementations may have that expectation). $\endgroup$ – Dave Nov 24 '20 at 12:41
0
$\begingroup$

YES

The typical way to do a ROC curve is to have probability outputs, set a threshold, and calculate the sensitivity and specificity.

y | prob
----------
0 | 0.2
0 | 0.3
1 | 0.7
1 | 0.8

At a threshold of 0, we classify everyting as $1$, so sensitivity of $1$ and specificity of $0$. At a threshold of $0.25$, we get $0,1,1,1$, so sensitivity of $1$ and specificity of $0.5$. Et cetera...

Plot those $(1, 1)$, $(0.5, 1)$ points to start building your ROC curve.

This idea of setting a threshold and calculating the sensitivity and specificity does not assume probability outputs, so let's do the same with non-probability outputs.

y | output
----------
0 | 2
0 | 3
1 | 7
1 | 8

At a threshold of 0, we classify everyting as $1$, so sensitivity of $1$ and specificity of $0$. At a threshold of $2.5$, we get $0,1,1,1$, so sensitivity of $1$ and specificity of $0.5$. Et cetera...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.