I'm trying to train various Keras models on Pascal VOC 2012 dataset. The particularity of this dataset is that there is a particular class used to label "ambiguous" regions. These pixels are meant to be ignored when computing the loss and accuracy. The pixel value on the annotation images for this class is $255$.

For this reason, I need to implement a custom categorical cross-entropy to ignore the pixels labelled $255$:

def restricted_categorical_crossentropy(y_true, y_pred, class_labels):
    def get_valid_probabilities_and_labels(annotation_batch_tensor,
        valid_labels_mask = tf.not_equal(annotation_batch_tensor,
                                         mask_out_class_label) # this is 255.
        valid_labels_indices = tf.where(valid_labels_mask)
        valid_batch_indices = tf.to_int32(valid_labels_indices)

        valid_labels_batch_tensor = tf.gather_nd(params=annotation_batch_tensor, indices=valid_batch_indices)
        valid_probabilities_batch_tensor = tf.gather_nd(params=logits_batch_tensor, indices=valid_batch_indices)

        return valid_labels_batch_tensor, valid_probabilities_batch_tensor

    valid_labels, valid_predictions = get_valid_probabilities_and_labels(

    cross_entropy = K.categorical_crossentropy(valid_labels, valid_predictions)

    # The number of elements is different during each step due to mask out regions: normalize entropy.
    return K.mean(cross_entropy)

When I compile my model with Keras' native K.categorical_crossentropy, I get values near $2.5$ and the model is able to slowly optimize that. When I use the custom function above, however, I get values like $11563889.6988$ and the model oscillates much more.

How does this make any sense granted

  1. my custom function uses K.categorical_crossentropy on a subset of pixels of the same batches;
  2. that exact same function works well when I'm not using Keras (I get values close to $2.5$ and lower)?

Also note that I preprocess the batches the exact same way in both situations (Keras vs. TensorFlow). Virtually nothing changed within the data pipeline except that I'm compiling a Keras model instead of running the session myself. I'm using the same TF records, same preprocessing function, same batching process.

Thanks in advance for your help!

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.