Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to something like an ROC curve. I'm interested in doing so because it's important for my project (image classification) to be able to adjust the classification thresholds.
$\begingroup$
$\endgroup$
9
-
2$\begingroup$ Are you sure you mean to use relu in the output layer? You do not need a bounded output to be able to do a ROC curve (though you might use a different approach to the code if you don’t have probability outputs), though relu would not be the standard choice for the output layer activation function. $\endgroup$– DaveCommented Nov 24, 2020 at 3:20
-
$\begingroup$ You can use sigmoid activation, and then apply the required threshold. $\endgroup$– Ashwin Geet D'SaCommented Nov 24, 2020 at 12:29
-
$\begingroup$ @AshwinGeetD’Sa Apply sigmoid when, instead of or after the relu? I still think the OP doesn’t mean to use relu in the output layer, though. $\endgroup$– DaveCommented Nov 24, 2020 at 12:34
-
$\begingroup$ @Dave, I meant instead of reLu. The OP mentions "convert the outputs from a ReLU", by this I assume, that OP wants to use ReLU instead of softmax. $\endgroup$– Ashwin Geet D'SaCommented Nov 24, 2020 at 12:37
-
1$\begingroup$ @AshwinGeetD’Sa Okay, so we’re in agreement that this is unorthodox and might be a mistake. // Mistake or not, I hope to get some time to be able to post an answer in the coming days. You don’t need probability outputs to do ROC (though particular software implementations may have that expectation). $\endgroup$– DaveCommented Nov 24, 2020 at 12:41
|
Show 4 more comments
1 Answer
$\begingroup$
$\endgroup$
YES
The typical way to do a ROC curve is to have probability outputs, set a threshold, and calculate the sensitivity and specificity.
y | prob
----------
0 | 0.2
0 | 0.3
1 | 0.7
1 | 0.8
At a threshold of 0, we classify everyting as $1$, so sensitivity of $1$ and specificity of $0$. At a threshold of $0.25$, we get $0,1,1,1$, so sensitivity of $1$ and specificity of $0.5$. Et cetera...
Plot those $(1, 1)$, $(0.5, 1)$ points to start building your ROC curve.
This idea of setting a threshold and calculating the sensitivity and specificity does not assume probability outputs, so let's do the same with non-probability outputs.
y | output
----------
0 | 2
0 | 3
1 | 7
1 | 8
At a threshold of 0, we classify everyting as $1$, so sensitivity of $1$ and specificity of $0$. At a threshold of $2.5$, we get $0,1,1,1$, so sensitivity of $1$ and specificity of $0.5$. Et cetera...
