Is regularization included in loss history Keras returns?

Question

I'm getting to know Keras. Right now, I'm testing with regularization and how to use them. Comparing the results of loss history for a training session with and without regularization, it seems to me that the loss history reported by Keras has the regularization term added to it, is that right?

When my model has no regularization term, the loss value starts from something less than 1 but when I retrain the model with regularization (L1L2), the same problem's loss value starts from 500. The only logical explanation I've got for it is that Keras is reporting loss value after it added the regularization term to it. And I believe the loss value without regularization is as valuable as with it, if not more valuable. Don't you think?

Answer 1

the regularisation values are by default computed in the loss and so you cannot see the regularisaiton values separately (as far as I know). For a brief explanation of an example using the cross-entropy loss with regularisation, have a look at the example in Michael Nielsen's book.

You can see what is returned and available by saving the results from calling model.fit().

You can train a model then also check out:

print(history.history.keys())

which will show you all available metrics for analysis after training the model.

You could perhaps define a custom function that compute the regularisation terms you are using, and execute that as either your own metric or as a callback function in a Keras model. If you use your own metric, those are by default recorded separately from the training/validation loss value.