AUC ROC in keras is different when using tensorflow or scikit functions.

Question

Two solutions for using AUC-ROC to train keras models, proposed here worked for me. But using tensorflow or scikit rocauc functions I get different results.

def auc(y_true, y_pred):
    auc = tf.metrics.auc(y_true, y_pred)[1]
    K.get_session().run(tf.local_variables_initializer())
    return auc

and

def auc(y_true, y_pred):
    return tf.py_func(roc_auc_score, (y_true, y_pred), tf.double)

Based on the history, it looks like both are being applied to train and validation. When I plot history metrics, tensorflow curve looks very smoothed compared to scikit.

Shouldn't I get about the same results using both functions?

Answer 1

No, you shouldn't have the same numbers. All depends on the additional parameters:

tf.metrics.auc(
    labels,
    predictions,
    weights=None,
    num_thresholds=200,
    metrics_collections=None,
    updates_collections=None,
    curve='ROC',
    name=None,
    summation_method='trapezoidal'
)

This means that this curve will have 200 points, so very smooth.

sklearn version doesn't have this kind of parameters:

roc_auc_score(y_true, y_score, average=’macro’, sample_weight=None, max_fpr=None)

The number of outputs depends on the curve and the number of points if I remember properly.