I have the following simple Bidirectional LSTM model for a binary classification task:

      MODEL = Sequential([
          Masking(mask_value = -1.0, input_shape = (None, 36)),
          Bidirectional(LSTM(units = 200, return_sequences = False, dropout = 0.20), input_shape = (None, 36), merge_mode = 'concat'),
          Dense(1, activation = 'sigmoid')

      MODEL.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = METRICS)

      HISTORY = MODEL.fit(Xtrain[train], ytrain[train], epochs = 32) # class_weight = weights, if using weights add this to .fit()

      PREDICTED = MODEL.predict(Xtrain[test])

I have been looking at some code where people calculate the AUC-ROC and AUC-PRC and what I see is them using the .predict_proba() function and also using only the positive class predictions to calculate the AUCs, like so: predicted = predicted[:,1].

I have calculated the AUCs like so:

from sklearn.metrics import auc, roc_acu_score

roc_auc_score(ytrain[test], PREDICTED)
fpr, tpr, _ = roc_curve(ytrain[test], PREDICTED) # to plot curve

precision, recall, thresholds = precision_recall_curve(ytrain[test], PREDICTED)
auc(recall, precision)

Given that my model predicts a single array of probabilities, am I calculating these metrics correctly?

Thanks in advance!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.