1
$\begingroup$

If I'm comparing performance between classification models on two different datasets should the number of samples per class, the number of classes, and features per sample be the relatively the same between the datasets?

Please let me know if you require any additional information.

$\endgroup$
2
  • $\begingroup$ Why are you doing this comparison? It sounds like “apples to oranges” and not very meaningful. In order to give a useful answer, it must be established what you want to learn from this comparison. $\endgroup$
    – Dave
    Dec 30, 2022 at 3:35
  • $\begingroup$ I am comparing the performance of diagnosis classification on two different kinds of biological data, mRNA data, and miRNA data to see which yields better classification performance. $\endgroup$ Dec 30, 2022 at 3:36

1 Answer 1

1
$\begingroup$

From the comments, OP aims to compare two different kinds of biological data in terms of which one allows the best classification performance.

To compare anything reliably, it's always best to keep invariant as many variables as possible and vary only the main objects being compared. So ideally, this comparison would be made between the two different kinds of data for the same subjects, same number and distribution of classes. In my opinion the number of features per sample does not have to be identical, since it's related to the 'kind of data'.

If this ideal comparison is not possible (for example if the data is not obtainable), it's crucial at the very least to compare the two options on the same classification problem. This means that the classes (especially number and distribution) must be identical, or at least very close. Otherwise one of the problems might be easier to solve than the other, so there would be no way to be sure that the result of the comparison is due to the 'kind of data' being compared or the different setting of the problems.

$\endgroup$
1
  • $\begingroup$ Thanks so much for your help! $\endgroup$ Dec 30, 2022 at 18:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.