For evaluating the classification of a highly imbalanced, there are several measures that you may consider. Remember, that in such a problem we would prefer a measure that is not biased towards one of the classes but gives similar importance to both classes.
Methods for evaluating classifiers could be divided into two types: singular assessment measures where the evaluation is given for a specific threshold (i.e. trade-off) and area-under-curve (AUC) analyses where a range of thresholds is considered.
The F-measure combines precision and recall. Precision is sensitive to data distribution as it combines values from the two rows of the confusion-matrix hence, as the difference between the numbers of negative and positive instances gets larger, the influence of the positive class (minority) gets smaller. Recall, on the other hand, is not sensitive to data distribution since it does not refer to the negative class at all. Affected by the precision, the F-measure must be sensitive to class distribution as well. The F-measure gives higher importance to the correct classification of positive instances, a change in the TNR value will influence the F-measure value less than a change in TPR. This is explained by the impact of the recall, which considers only the correct classification of positive class instances.
The Geometric Mean (G-measure, GM) combines TPR and TNR, denoting the accuracy rates measured on both classes separately. The GM is independent of the class sizes, indicating the relative part of correctly classified instances in each class regardless of the degree of imbalance. That is, the GM is not influenced by the imbalance between the classes, The GM relates to the performance on both classes with the same importance since it is a harmonic mean of the TPR & TNR and their effect is symmetrical. Higher values of GM are achieved when the accuracies on both classes are high and similar. For example, a higher GM value is accomplished when accuracies on both classes are equal to 0.5 than when one is 0.6 and the other 0.4.
The ROC-AUC is based on the TPR and FPR, which is 1-TNR, and hence have the same reasoning for suiting such problems as the GM.
The PR-AUC, precision-recall curve, are also useful for highly imbalanced data and also is robust for changes in the balance of classes (addition of samples). I've found this article very useful for understanding AUC measures and the benefits of each one of them.
The AUC analysis provides a more generic evaluation however, in most cases it will not give the best solution when a specific trade-off is to be considered. The AUC is only relevant if your classifier output is not discrete but continues.
For more reading and comparisons you may use Prof. Haibo He lectures which are very comprehensive and useful.
