# Probability Calibration : role of hidden layer in Neural Network

I try a simple Neural Network (Logistic Regression) to play with Keras. In input I have 5,000 features (output of a simple tf-idf vectorizer) and in the output layer I just use a random uniform initialization and an $$\alpha = 0.0001$$ for $$L_{2}$$ regularization with a sigmoid activation function (so something pretty standard).

I use an Adam optimizer and the loss function is the binary cross-entropy.

When I display the probability for both classes I end up with something like this :

After that I try to add a hidden layer with a $$Relu$$ activation function, 64 nodes and again the same parameters for regularization & initialization. -- EDIT -- Here for the output layer I keep exactly the same parameter (and $$sigmoid$$ activation function) as in the previous NN. -- END OF EDIT --

But now when I plot the probability distribution for both classes I end up like this :

And cleary I didn't get why by adding a hidden layer the probability are push to 0 or 1 ?

Have you any references that I can read so I can understand the math behind ? It could be great!

Moreover sometimes (with a more "deeper" neural network for another application) I get the same plot as the second one but this time the predicted probabilities are between $$[0.2; 0.8]$$. The probabilities are push to some "values" but those values are more "centered". Not sure it is clear. And here again I don't understand what is the phenomena behind this ? How can I "debug" it to see in my architecture of my Neural Network what is the cause ?

In addition How can I "tweak" my ANN to have a "perfect" calibration plot (like in the scikit webpage : https://scikit-learn.org/stable/modules/calibration.html) ?

Thank you in advance for every answer that can enligthen me :)

It makes sense that a network with ReLU activations produces worse probabilities than a sigmoid activation when you think about how these activation functions interact with the unit in the output layer of the network. Sigmoid scales the output to be in the range $$[0, 1]$$, so the values going into the final sigmoid classification unit will be fairly low. Meanwhile, the ReLU-based network has the potential for very high activation magnitudes, which will skew the estimated probabilities to either end of the spectrum.

Having said that, calibration of neural networks is not very well understood in general. A paper was published last year at ICML (https://arxiv.org/pdf/1706.04599.pdf) which showed that often neural nets are poorly calibrated, but it doesn't really explain why this is the case. However, they did investigate factors that affect calibration, and show that deeper, wider networks can often be less well calibrated than their shallower, narrower counterparts. They also show that applying $$L_2$$ regularisation can improve calibration.

Given this information, I'm surprised that your (fairly shallow and narrow) network is so poorly calibrated. It might be overfitted to NLL, so I would try applying heavier regularisation or training for fewer epochs. If this doesn't work, try implementing temperature scaling, which is described in the paper I linked above. They provide a PyTorch implementation here, so it shouldn't be too difficult hopefully to translate into Keras.

• Thank for your answer. Here I don't get the point with the Relu activation since in my output layer it is always a sigmoid function ? But it is interesting that you too are surprised by the result (it is not a deep NN) and I think (I will check again) than there is no overfitting. I will look at the paper (thank for the reference) & at the temperature scaling. – BimBimBap Nov 20 '18 at 9:08
• I clarify my first post for the output layer (they are always the same, the only change is the add of a hidden layer) – BimBimBap Nov 20 '18 at 9:24

I spent way too long attempting to calibrate my keras probability outputs. It turned out to be very simple.

The isotonic model does a great job (so much so it risks overfitting)

from sklearn.isotonic import IsotonicRegression
ir = IsotonicRegression()
ir.fit(results.pred,results.act)
results['iso'] = ir.predict(results.pred)


This blog post has more details and some useful code for calibration plots. And best thing is the sum of my probabilities adds up to same as the number of cases.

Obviously you should validate this by training the ir model on your training set and then apply it to your networks predictions on the test data. I've been very happy with the performance measured by Brier scores and AUC.

You'll see references to Platt scaling. As far as I can tell isotonic scaling is superior on large datasets.

• Linked scikit docs say "Probability calibration should be done on new data not used for model fitting". I guess I interpret that to mean that the isotonic regression model might be better trained on e.g. the validation set instead of the training set. What do you think? – J Trana Aug 28 '19 at 3:06