1
$\begingroup$

I know when model is made to predict a float value, a common approach to report the models validation is using k-fold technique and calculating the average of all folds accuracy (here is a similar question).

Now suppose that my model is a classifier and each fold outputs a confusion matrix. how can i combine confusion matrixs

$\endgroup$

1 Answer 1

1
$\begingroup$

You can just sum all the cells across folds: for every true class $T_i$ and every predicted $P_j$, the number is the sum of this cell $(T_i,P_j)$ for fold 1, fold 2, .., fold N.

This is because the confusion matrix is made with the test set for every fold, and by construction the union of all the test sets across folds is equal to the full dataset. This way one can report performance for the full dataset, without data leakage of course.

A confusion matrix is not an evaluation measure though.

$\endgroup$
2
  • $\begingroup$ is it a valid method? i mean do you know a paper or book which proposes summation? @Erwan $\endgroup$
    – morteza
    Sep 29, 2022 at 12:02
  • 1
    $\begingroup$ @morteza Papers very rarely publish confusion matrices, they publish scores based on evaluation measures like the F1-score. I explained in my answer why it's valid: if ones understands how k-fold cross-validation works, it follows that the total confusion matrix is obtained by summing the confusion matrices obtained on the different test sets. It's a bit trivial, so I don't think there is any resource explaining it. $\endgroup$
    – Erwan
    Sep 29, 2022 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.