2
$\begingroup$

enter image description here

here precision at threshold 0.85 > precision at threshold 0.90. shouldnt it be the other way round? increasing threshold will reduce False positive and precision will be greater than before?

$\endgroup$
6
$\begingroup$

here precision at threshold 0.85 > precision at threshold 0.90. shouldnt it be the other way round? increasing threshold will reduce False positive and precision will be greater than before?

Precision is $\frac{\text{TP}}{\text{TP}+\text{FP}}$

Both $\text{TP}$ and $\text{FP}$ are reduced when you increase the threshold. If both decrease in proportion to the current precision (i.e. they are spread evenly at each confidence value), then precision will remain the same. Most models on most datasets will tend to increase precision as the threshold increases, at least initially (e.g. moving from 0.5 to 0.6) as false positives may commonly be found as uncertain edge cases with low confidence, i.e. false positives tend to occur more frequently at low confidence, so increasing threshold will exclude a higher ratio of false positives than true positives than the current precision. However, there is no guarantee of that.

The value of precision will vary in practice depending on what the model predicted for each example. If you have a cluster of highly confident false positives, they can cause precision to drop as threshold grows, until they get excluded. The most extreme example would be where the most confident classification is incorrect, in which case the highest possible threshold will score zero precision.

$\endgroup$
1
  • $\begingroup$ Thank you for the explanation :) $\endgroup$
    – dataguy
    May 2 '20 at 0:24
0
$\begingroup$

If you look at ROC curve you get to see a different picture ROC-AUC curve

enter image description here

Uptill point A TPR is increasing faster than FPR so that means precision should be higher here. After point A TPR is increasing at slower rate than FPR and so Precision should start decreasing, after point B the precision should further decrease.

Here in ROC from left to right threshold is increasing, so as threshold increases, initially precision also increases but then later starts to drop. Recall continues to increase from left to right.

But my doubt is if really threshold is increasing in ROC from left to right, most of the online notes and text just says "ROC is plotted against different threshold values" It doesn't say whether threshold is increasing from left to right or decreasing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.