ROC-AUC loss for GRU Model: Cannot use tflearn's loss in keras

Question

I am trying to use tflearn.objectives.roc_auc_score as a loss function for a GRU network in Keras but I get the following error:

> ValueError: An operation has `None` for gradient. Please make sure
> that all of your ops have a gradient defined (i.e. are
> differentiable). Common ops without gradient: K.argmax, K.round,
> K.eval.

This is surprising as the implementation is apparently based on an approximation that is supposed to be differentiable.

For your reference, here is the code from the tflearn Github:

def roc_auc_score(y_pred, y_true):
    """ ROC AUC Score.
    Approximates the Area Under Curve score, using approximation based on
    the Wilcoxon-Mann-Whitney U statistic.
    Yan, L., Dodier, R., Mozer, M. C., & Wolniewicz, R. (2003).
    Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic.
    Measures overall performance for a full range of threshold levels.
    Arguments:
        y_pred: `Tensor`. Predicted values.
        y_true: `Tensor` . Targets (labels), a probability distribution.
    """
    with tf.name_scope("RocAucScore"):

        pos = tf.boolean_mask(y_pred, tf.cast(y_true, tf.bool))
        neg = tf.boolean_mask(y_pred, ~tf.cast(y_true, tf.bool))

        pos = tf.expand_dims(pos, 0)
        neg = tf.expand_dims(neg, 1)

        # original paper suggests performance is robust to exact parameter choice
        gamma = 0.2
        p     = 3

        difference = tf.zeros_like(pos * neg) + pos - neg - gamma

        masked = tf.boolean_mask(difference, difference < 0.0)

return tf.reduce_sum(tf.pow(-masked, p)) 

Answer 1

This worked for me when I changed the declaration of the function:

def roc_auc_score(y_true, y_pred)

instead:

def roc_auc_score(y_pred, y_true)

I defined it wrong and it worked, when I fixed it I got the same error as you do. I don't really understand your code logic, but you can if this works for you.

Answer 2

for me at least this problem was caused outside the above function because I was using tf.argmax to obtain y_pred which is not differentiable (and also incorrect). I replaced by y_pred[:, 1] which solved the problem.