2
$\begingroup$

I am new in this area. I am facing some issues while comparing the algorithms using statistical test. I have following result of Gmean of some classification algorithm. Abalone, Balance-scale, Car, Chess are the datasets here and ROS, RUS, RFS, NoS are the algorithms.

If I want to compare which dataset is better than others using some statistical test like, t-test, Friedmen test, Wilcoxon test etc then is it possible that I can compare the algorithm using the following table?

                  ROS       RUS     RFS     NoS
Abalone           0.003     0.0036  0.0039  0
Balance-scale     0.8858    0.8065  0.8966  0.9417
Car               0.9191    0.7216  0.9056  0.9094
Chess             0.4912    0.1973  0.5084  0.1438

If anyone has any idea about it please help me. Or you can share any references from where I can find the solutions. I studied these statistical test, null hypothesis, p-value etc but couldn't understand whether it is possible or not to compare these algorithms using Gmean.

***G-mean = Geometric Mean, used to evaluate the performance of multiclass classifiers

Thanks in advance.

$\endgroup$
0
$\begingroup$

The performance of machine learning algorithms is not commonly evaluated with null hypothesis significance testing (NHST).

Machine learning performance is evaluated with performance on a hold-out data (e.g., validation or test), regardless of the evaluation metric.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.