How to improve the evaluation score for highly imbalanced dataset?

Question

I have trained my BERT model(bert-base-cased) to detect toxic comments. I used the Toxic Comment Classification Challenge dataset from the Kaggle. My accuracy is 98% and the AUROC for various sub-classes is above 90%. However, my Precision, Recall, and F1 score is less. The scores are shown in the image Evaluation Scores. The dataset is highly imbalanced. The ratio of clean comments is way higher than the toxic comments. Any suggestions to improve the evaluation scores?

Here's the final score

           precision  recall  f1-score   support

micro avg 0.61 0.85 0.71 1743
macro avg 0.56 0.69 0.61 1743
weighted avg 0.64 0.85 0.72 1743
samples avg 0.08 0.09 0.08 1743

Answer 1

For imblanced data I recommend using false positive ratio instead of precision. In contrast to recall, precision is affected by the positive negative ratio, false-positive ratio (or 1 minus this number) is not affected by the data distribution, only by your model.

Answer 2

For imbalanced data, I suggest you use "ROC, AUC Curve" or F1 score. Check out my article "What Is Balanced And Imbalanced Dataset?" -> https://medium.com/analytics-vidhya/what-is-balance-and-imbalance-dataset-89e8d7f46bc5