# How to improve the evaluation score for highly imbalanced dataset?

I have trained my BERT model(bert-base-cased) to detect toxic comments. I used the Toxic Comment Classification Challenge dataset from the Kaggle. My accuracy is 98% and the AUROC for various sub-classes is above 90%. However, my Precision, Recall, and F1 score is less. The scores are shown in the image Evaluation Scores. The dataset is highly imbalanced. The ratio of clean comments is way higher than the toxic comments. Any suggestions to improve the evaluation scores?

Here's the final score

           precision  recall  f1-score   support


micro avg 0.61 0.85 0.71 1743
macro avg 0.56 0.69 0.61 1743
weighted avg 0.64 0.85 0.72 1743
samples avg 0.08 0.09 0.08 1743

For imblanced data I recommend using false positive ratio instead of precision. In contrast to recall, precision is affected by the positive negative ratio, false-positive ratio (or 1 minus this number) is not affected by the data distribution, only by your model.

• Thank you @Amit, I'll try using it for my evaluation. Aug 20 at 15:06