I have built a model using the xgboost package (in R), my data is unbalanced (5000 positives vs 95000 negatives), with a binary classification output (0,1).
I have performed cross validation with the evaluation metric
AUC Area under the ROC curve which I now believe to be wrong since this is better used for balanced data sets.
I analyse the final results of the model using the Area Under the Precision Recal curve (AUPRC) and the Matthews Correlation Coefficient (MCC), however I now believe that I should have been evaluating the cross validation models with the AUPRC and MCC also and completely forget about the AUROC.
I cannot find much in the literature which uses CV with the evaluation metric of AUPRC and MCC.
I just want to make sure that I am thinking correctly and that my previous evaluation method is wrong and the AUPRC / MCC would be a better way to go.