Skip to main content
Data Science

Return to Question

Added missing tags, fixed intelligibility
Source Link
edit approved Dec 22, 2020 at 8:51
Shayan Shafiq
edit approved Dec 22, 2020 at 8:51
Shayan Shafiq
  • 1k
  • 4
  • 13
  • 24

I have a multi-class classification problem and. I am primarily using macro-average F1 measure to evaluate the performance of models.

I and want to verify if the results are statistically significant.

  I have the results ifof two classifiers on the same train/test-set.  (paired observations).

Some sources suggest to use McNemar’s test for Binarybinary classification task, however. However, is there any generalization of McNemar’s test for multi-class classification problem.? If so, what would be the appropriate procedure to carry out these tests.?

I have a multi-class classification problem and I am primarily using macro-average F1 measure to evaluate the performance of models.

I want to verify if the results are statistically significant.

  I have the results if two classifiers on the same train/test-set.(paired observations).

Some sources suggest to use McNemar’s test for Binary classification task, however, is there any generalization of McNemar’s test for multi-class classification problem. If so, what would be the appropriate procedure to carry out these tests.

I have a multi-class classification problem. I am primarily using macro-average F1 measure to evaluate the performance of models and want to verify if the results are statistically significant. I have the results of two classifiers on the same train/test-set  (paired observations).

Some sources suggest to use McNemar’s test for binary classification task. However, is there any generalization of McNemar’s test for multi-class classification problem? If so, what would be the appropriate procedure to carry out these tests?

Source Link
Ashwin Geet D'Sa
Ashwin Geet D'Sa
  • 1.2k
  • 2
  • 10
  • 20

What is the appropriate statistical significance test for multi-class classification?

I have a multi-class classification problem and I am primarily using macro-average F1 measure to evaluate the performance of models.

I want to verify if the results are statistically significant.

I have the results if two classifiers on the same train/test-set.(paired observations).

Some sources suggest to use McNemar’s test for Binary classification task, however, is there any generalization of McNemar’s test for multi-class classification problem. If so, what would be the appropriate procedure to carry out these tests.