Calibrate the predicted class probability to make it represent a true probability?

Question

Let's say that we have a simple binary classification model (a neural network -- NN) for classifying input images as "dog" ($y=1$) or "not dog" ($y=0$). Let's assume that the NN has one "sigmoid output" $\hat{y}$. After training, we feed an image of a dog and the output of the NN is e.g. $\hat{y} = 0.8$. As I understand it, this value of "0.8" is the probability ("confidence") that the particular image is a "dog", according to the model, and the prediction is that "it's a dog" (because 0.8 > 0.5).

But is the $\hat{y}$ really a (true) probability? What I mean is: if we feed the NN with many different images, and say for 100 of them the model gives the same $\hat{y} = 0.8$. For this $\hat{y}$ to be a true probability, I would expect 80 of these images to actually be dogs and 20 not dogs. To check this we would look at the $y$ (assuming that they exist). In the case when all the 100 images are actually dogs, our model made correct prediction (since it correctly labeled all of them as dogs, because 0.8 > 0.5 for all of the 100 images), but the $\hat{y}$ is not a true probability prediction, because it should be 1. This is actually the accuracy, $100/100 = 1$. On the other hand, if out of 100 images 80 are actually dogs and 20 are not dogs, then the model prediction has accuracy $80/100 = 0.8$, because it incorrectly classified all 100 images as dogs, again, but only 80 are actually dogs. However now the $\hat{y} = 0.8$ represents the true probability. In other cases, the accuracy can be e.g. 0.9, meaning that the predicted $\hat{y} = 0.8$ is not a correct probability, but is closer to it than in the first case.

So, first of all, is what I am saying correct? Second, does the "calibration" fix this issue, i.e. making $\hat{y}$ = accuracy allows the $\hat{y}$ to be interpreted as true probability?

Answer 1

"Not all classification models are naturally probabilistic" is the first line of this chapter [1] . This means that some models do output probabilities, and some do not. This is highly dependant on the model structure as well as of the objective function that it uses.

In particular, for neural network, a commonly used objective function is Cross Entropy. And to my understanding, Cross Entropy does try to approach the true distribution of the target variable, conditionned on the input variable. This would mean that for a hundred similar cases $X_{similar}$, that are similar in terms of the information that the model is capable to understand, if 80% of those cases are dogs, the predicted score will be a descent approach of the probability you want to estimate: $ \hat{y} \approx p(dog|X_{similar}) = 80/100$.

For other models with less "statistical" behaviour, such as Decision Tree, SVM, of models taken into a Boosting or Bagging process, the output distribution ($\hat{y}$) is skewed. In this binary classification setting, these models would try to give a score as close as possible of the target value. In the case at hand, they would consider the $X_{similar}$ as a group, and the best decision for that group in terms of accuracy beeing to classify them all as dogs, $\hat{y}$ would be skewed toward 1 value.

So, my take on this :

• If your goal is to have accurate probabilities, use a model that directly outputs probabilities. If the probabilities are "perfect", a cutoff value of 0.5 would provide the best accuracy (i.e. $\hat{y}\geq{0.5}$ as a decision rule)

• If your goal is to have a good classifier in terms of accuracy, use the model that... has the best accuracy. Note that in this case, the best cutoff value may not be 0.5 (it can even be far from it).

• If you have both those goals, well, I do not have the answer as this is data and model dependant. If your scores are not probabilities, calibration should make them look more like it indeed. See Calibration in classification for a list of calibration methods.

[1] Wikipedia - Probabilistic Classification - Probability calibration

[2] Wikipedia - Calibration (statistics) - In Classification