What inference can we draw from the frequency distribution of thresholds?

I've the probability scores of positive class of two models. The frequency distribution of those probability scores(thresholds) are like this

Model #1

Model #2

It's a binary classification problem. And the thresholds obtained are from the test data. The stats in the confusion matrix are nearly same for two models. Can we say the Model #2 is doing well in separating classes? What other conclusion we can draw from the frequency of the thresholds.

I'd say the model 1 is performing really well. If you can use different colors while plotting for positive and negative classes, then you should be able to see the difference. When you are trying to do binary classification, the distributions of negative and positive classes should be dipping in the mid region where as Model 2 is the opposite.

• But, I think no need to color it, as we know the observation >0.5 are positively predicted, and rest all are negatively predicted. Can you please explain more on why "dipped in the mid region" is expected? Jan 16, 2020 at 16:23
• You don't have to colour it but it helps to differentiate between classes. Dipping in the mid region is good to have because all the negative classes should be skewed to the left and the positive classes should be skewed to the right which means that the model is able to classify well. Jan 16, 2020 at 16:29
• Ok, I'm thinking in a diagnostic manner, in model #1 when we change the threshold from 0.5 to 0.3 nothing much changed in confusion matrix(as it less densed around 0.5). So for the model which has good class separation, the frequency of threshold will look like the one in the model #1. Am I correct? Jan 16, 2020 at 16:36
• Yes absolutely. Jan 16, 2020 at 16:39