Probability Calibration : role of hidden layer in Neural Network

Question

I try a simple Neural Network (Logistic Regression) to play with Keras. In input I have 5,000 features (output of a simple tf-idf vectorizer) and in the output layer I just use a random uniform initialization and an $\alpha = 0.0001$ for $L_{2}$ regularization with a sigmoid activation function (so something pretty standard).

I use an Adam optimizer and the loss function is the binary cross-entropy.

When I display the probability for both classes I end up with something like this :

After that I try to add a hidden layer with a $Relu$ activation function, 64 nodes and again the same parameters for regularization & initialization. -- EDIT -- Here for the output layer I keep exactly the same parameter (and $sigmoid$ activation function) as in the previous NN. -- END OF EDIT --

But now when I plot the probability distribution for both classes I end up like this :

And cleary I didn't get why by adding a hidden layer the probability are push to 0 or 1 ?

Have you any references that I can read so I can understand the math behind ? It could be great!

Moreover sometimes (with a more "deeper" neural network for another application) I get the same plot as the second one but this time the predicted probabilities are between $[0.2; 0.8]$. The probabilities are push to some "values" but those values are more "centered". Not sure it is clear. And here again I don't understand what is the phenomena behind this ? How can I "debug" it to see in my architecture of my Neural Network what is the cause ?

In addition How can I "tweak" my ANN to have a "perfect" calibration plot (like in the scikit webpage : https://scikit-learn.org/stable/modules/calibration.html) ?

Thank you in advance for every answer that can enligthen me :)

Answer 1

It makes sense that a network with ReLU activations produces worse probabilities than a sigmoid activation when you think about how these activation functions interact with the unit in the output layer of the network. Sigmoid scales the output to be in the range $[0, 1]$, so the values going into the final sigmoid classification unit will be fairly low. Meanwhile, the ReLU-based network has the potential for very high activation magnitudes, which will skew the estimated probabilities to either end of the spectrum.

Having said that, calibration of neural networks is not very well understood in general. A paper was published last year at ICML (https://arxiv.org/pdf/1706.04599.pdf) which showed that often neural nets are poorly calibrated, but it doesn't really explain why this is the case. However, they did investigate factors that affect calibration, and show that deeper, wider networks can often be less well calibrated than their shallower, narrower counterparts. They also show that applying $L_2$ regularisation can improve calibration.

Given this information, I'm surprised that your (fairly shallow and narrow) network is so poorly calibrated. It might be overfitted to NLL, so I would try applying heavier regularisation or training for fewer epochs. If this doesn't work, try implementing temperature scaling, which is described in the paper I linked above. They provide a PyTorch implementation here, so it shouldn't be too difficult hopefully to translate into Keras.

Answer 2

I spent way too long attempting to calibrate my keras probability outputs. It turned out to be very simple.

The isotonic model does a great job (so much so it risks overfitting)

from sklearn.isotonic import IsotonicRegression
ir = IsotonicRegression()
ir.fit(results.pred,results.act)
results['iso'] = ir.predict(results.pred)

This blog post has more details and some useful code for calibration plots. And best thing is the sum of my probabilities adds up to same as the number of cases.

Obviously you should validate this by training the ir model on your training set and then apply it to your networks predictions on the test data. I've been very happy with the performance measured by Brier scores and AUC.

You'll see references to Platt scaling. As far as I can tell isotonic scaling is superior on large datasets.