I would like to compare two overall accuracy statistics (of a Random Forest classifier).

My Data:

  • two samples with each containing 25 features and one categorical class variables (9 different categories),
  • N ~100 for each category = N ~ 900 total, for both datasets.
  • I've used RF with 10-fold CV in each dataset with reasonable overall accuracy
    • 89% in Dataset1
    • 86% in Dataset2.

Can I calculate an effect size for the difference? Is there a common practice to do so? Since I have no measure for the deviation I can't calculate d. Would it be appropriate to calculate an Odds-Ratio?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.