3
$\begingroup$

Two solutions for using AUC-ROC to train keras models, proposed here worked for me. But using tensorflow or scikit rocauc functions I get different results.

def auc(y_true, y_pred):
    auc = tf.metrics.auc(y_true, y_pred)[1]
    K.get_session().run(tf.local_variables_initializer())
    return auc

and

def auc(y_true, y_pred):
    return tf.py_func(roc_auc_score, (y_true, y_pred), tf.double)

Based on the history, it looks like both are being applied to train and validation. When I plot history metrics, tensorflow curve looks very smoothed compared to scikit.

Shouldn't I get about the same results using both functions?

$\endgroup$
4
$\begingroup$

No, you shouldn't have the same numbers. All depends on the additional parameters:

tf.metrics.auc(
    labels,
    predictions,
    weights=None,
    num_thresholds=200,
    metrics_collections=None,
    updates_collections=None,
    curve='ROC',
    name=None,
    summation_method='trapezoidal'
)

This means that this curve will have 200 points, so very smooth.

sklearn version doesn't have this kind of parameters:

roc_auc_score(y_true, y_score, average=’macro’, sample_weight=None, max_fpr=None)

The number of outputs depends on the curve and the number of points if I remember properly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.