# ROC curve and optimal threshold

I am doing a practice problem predicting a binary outcome. I have plotted an ROC curve and found the optimal threshold percentage to call future predicted observations a 1. I see that this threshold always matches the percentage of observations equal to 1 in my original data. Is there any conceptual explanation for this?

The training data contains a proportion $$p$$ of instances labelled 1. From the ROC plot you can see all the possible values for setting the threshold at a certain level and the resulting performance; for every possible level you can calculate the corresponding proportion $$q$$ of instances predicted as 1:
• if $$q$$ is much lower than $$p$$, then the system predicts many 0s, so there are many false negative errors and that makes the recall lower. Precision is high in this case.
• if $$q$$ is much higher than $$p$$, then the system predicts many 1s, so there are many false positive errors and that makes the precision lower. Recall is high in this case.
I assume that you optimize on the F1-score right? The fact that the F1-score is based on the product of the precision and recall means that both values need to be reasonably high, otherwise the F1-score drops. As seen above, having very different values for $$p$$ and $$q$$ will cause either the precision or recall to be low. Therefore the optimal F1-score is achieved when $$q$$ is close to $$p$$.