I would like to compare two overall accuracy statistics (of a Random Forest classifier).
- two samples with each containing 25 features and one categorical class variables (9 different categories),
- N ~100 for each category = N ~ 900 total, for both datasets.
- I've used RF with 10-fold CV in each dataset with reasonable overall accuracy
- 89% in Dataset1
- 86% in Dataset2.
Can I calculate an effect size for the difference? Is there a common practice to do so? Since I have no measure for the deviation I can't calculate d. Would it be appropriate to calculate an Odds-Ratio?