What type of metrics I should use to evaluate my classification models, given that I have two imbalanced multi-class datasets (21 and 16 classes, respectively) where all classes have equal importance?
I am somehow convinced with macro-averaged-based metrics choice such as macro F1 and macro TNR, ...etc. Are macro-averaged-based metrics suitable for my problem based on the aforementioned inputs?
Yes, a macro-average measure is the standard choice in this context: a macro-average score is simply the mean of the individual score for every class, thus it treats every class equally.
With an strongly imbalanced dataset, this means that a small class which has only a few instances instances in the data is given as much weight as the majority class. Since the former is generally harder for a classifier to correctly identify, the macro-average performance value is usually lower than a micro-average one (this is normal of course).
