# AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?

I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows:

              precision    recall  f1-score   support

airplane       0.93      0.90      0.91      1000
automobile       0.93      0.96      0.95      1000
bird       0.88      0.87      0.87      1000
cat       0.86      0.72      0.79      1000
deer       0.88      0.91      0.89      1000
dog       0.88      0.81      0.84      1000
frog       0.83      0.97      0.89      1000
horse       0.94      0.94      0.94      1000
ship       0.95      0.93      0.94      1000
truck       0.90      0.95      0.92      1000

accuracy                           0.90     10000
macro avg       0.90      0.90      0.90     10000
weighted avg       0.90      0.90      0.90     10000


The code where I calculate the ROC Curve and AUC is below:

def assess_model_from_pb(model_file_path: Path, xtest: np.ndarray, ytest: np.ndarray, save_plot_path: Path):

class_labels = ['airplane','automobile','bird','cat','deer','dog','frog','horse','ship','truck']
model = load_model(model_file_path) # load model from filepath
feature_extractor = Model(inputs = model.inputs, outputs = model.get_layer('dense').output) # extract dense output layer (will be softmax probabilities)
y_score = feature_extractor.predict(xtest, batch_size = 64) # one hot encoded softmax predictions
ytest_binary = label_binarize(ytest, classes = [0,1,2,3,4,5,6,7,8,9]) # one hot encode the test data true labels
n_classes = y_score.shape[2]

fpr = dict()
tpr = dict()
roc_auc = dict()
# compute fpr and tpr with roc_curve from the ytest true labels to the scores
for i in range(n_classes):
fpr[i], tpr[i], _ = roc_curve(ytest_binary[:, i], y_score[:, i])
roc_auc[i] = auc(fpr[i], tpr[i])

# plot each class  curve on single graph for multi-class one vs all classification
colors = cycle(['blue', 'red', 'green', 'brown', 'purple', 'pink', 'orange', 'black', 'yellow', 'cyan'])
for i, color, lbl in zip(range(n_classes), colors, class_labels):
plt.plot(fpr[i], tpr[i], color = color, lw = 1.5,
label = 'ROC Curve of class {0} (area = {1:0.3f})'.format(lbl, roc_auc[i]))
plt.plot([0, 1], [0, 1], 'k--', lw = 1.5)
plt.xlim([-0.05, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve for CIFAR-10 Multi-Class Data')
plt.legend(loc = 'lower right', prop = {'size': 6})
fullpath = save_plot_path.joinpath(save_plot_path.stem +'_roc_curve.png')
plt.savefig(fullpath)
plt.show()


I suppose I am just confused on how my AUC can be near 1 when my precision and recalls are not perfect. I understand that many thresholds are used to determine what is a positive class and a negative class. For example towards the beginning of the curve, if the threshold is very high (say around .99999), how is it that my tpr is near 1? Is it solely the fact that at that threshold I will only be giving positive classifications for the most absolutely high softmax probabilities?

Would just like a bit more explanation or intuition on the topic to make sure I am not doing something incorrectly.

• Please delete the copy you asked earlier on stack overflow: stackoverflow.com/q/64126632/333599 Sep 30 '20 at 5:57
• You could translate your ROC curve into precision/recall if you were considering a multi-label problem. I imagine the accuracies are from a multi-classification evaluation, which means that even though there is a threshold that perfectly separates images with cars and images without cars, sometimes another class is even more positive (maybe truck for instance). Sep 30 '20 at 12:10
• You could check the threshold value of the sweet spot, if it is above 0.5, my guess would be wrong Sep 30 '20 at 12:17