1
$\begingroup$

When Keras prints the validation loss, it prints the total loss value (regularization loss + fitting loss).

How can I get the two separately? I'm specifically looking to obtain value for regularization loss.

$\endgroup$
0
$\begingroup$

By default, most loss functions only return one scalar value. Some will also give you a 'per-sample loss in the batch' value. If you want other loss categories, you can use the add_loss() to add, compute and keep track of different types of losses in different layers.

Refer to https://keras.io/api/losses/ for more details.

$\endgroup$
0
$\begingroup$

If you just want the overall regularization loss, you could make a call back function calculates the loss function on the sample without regularization and subtract it from the evaluated loss function. How to write you own callbacks.

Let's say you have a layer with kernel_regularize=tf.keras.regularizers.l2(1.5). The regularization loss will then be 1.5*tf.reduce_sum(W**2), where W is are the weights without the bias. For an L1 loss, the absolute value is taken instead of the squared value. The regularization loss at each layer will linearly add to the overall loss function.

class LossAndErrorPrintingCallback(keras.callbacks.Callback):
    def on_train_batch_end(self, batch, logs=None):
        print("For batch {}, loss is {:7.2f}.".format(batch, logs["loss"]))

    def on_test_batch_end(self, batch, logs=None):
        print("For batch {}, loss is {:7.2f}.".format(batch, logs["loss"]))

    def on_epoch_end(self, epoch, logs=None):
        print(
            "The average loss for epoch {} is {:7.2f} "
            "and mean absolute error is {:7.2f}.".format(
                epoch, logs["loss"], logs["mean_absolute_error"]
            )
        )
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.