# Which metric should I use for classifying an imbalace data with fewer labels for the negative class?

From reading, I understand that when we have fewer positive class labels, it is better to use precision or recall as the evaluation metric. Which metric should I use when we have fewer negative samples?

I'm looking for an approach other than switching the labels.

Problem setting: I'm developing parametrized fragility functions for predicting damage to a structure (for example trees). An example of fragiltiy function is here The fragility function will estimate the probability of exceeding a damage state given some parameters (say wind load). The damage state can be expressed in terms of damage ratio (0-1 with 1 being fully damaged). Now, we are interested in estimating the probability of exceeding a damage ratio given features. To elaborate, the probability of any damage would be P(Damage_ratio>0.0|features). Logistic regression can be used to learn this curve from data after categorizing 0-1 continuous damage ratio to damaged (- class)/no damage (+ class) for a particular threshold. Now, as we move from the threshold from 0 to 1, the dataset transforms from imbalanced data dominated by damaged cases to a balanced state and then finally to another imbalanced data dominated by non-damage case.

Now, when learning the model, 'AUR-ROC' performs really well when the data is balanced. Precision performs well when the data is imbalanced with more no-damage case (P(Damage_ratio>0.1|features)). These metrics don't do well for the case with few negative case (P(Damage_ratio>0.9|features)). I tried switching the label with very limited success. Are there any other 'metrics' that perform well in an imbalanced data setting?

• Why do you need the minority class to be called "negative"? There might be a good reason but you should mention it imho. What is the task? Nov 10 '20 at 21:46
• Thanks for adding details, it looks to me like the problem is not with the evaluation measure: the smaller the minority class, the harder it is for the classifier, so it's normal that the performance decreases. Changing the evaluation measure would be like hoping that changing the thermometer will make the fever disappear ;) However have you considered treating the problem as a regression task, i.e. predicting directly the "damage ratio" value instead of a class? Nov 10 '20 at 23:42
• :) Thank @Erwan. I agree with you; for extreme damage ratios, I have only a few samples to learn. My intention was to get the best result possible given the constraints and one way we could do that is to use appropriate metrics for training (AUC-ROC vs AUC-PR). Yes, I am developing a regression model in addition to the classification model. Thanks a lot.
– PPR
Nov 11 '20 at 0:04