0
$\begingroup$

I am using two classification algorithms in Weka i.e. Logistic Regression and Naive Bayes and want to know which algorithm has better performance? I need a statistical test like Bayesian, so which type of Bayesian test I need to perform?

Let me explain a bit. I have to perform regression based prediction on two different algorithms. The output is absolute residual I.e difference between actual and predicted values. For instance, algorithm A has results like 0.3, 0.12, 0.01, 0.7, 0.32 etc and algorithm B has values like 0.1, 0.9, 0.56, 0.32, 0.01 etc. The total instances could be more than 100. Now in weka, I got the results that algorithm A is better than B. Now I want to perform a statistical test that these results are not equivalent. In case of Wilcox test, for instance if I get p value less than 0.05, I say A is significantly better than B. How can I do this using baysian test or any other test which does not have p values?

Improve this question
$\endgroup$
  • $\begingroup$ Please clarify what you're trying to do: what kind of data do you have, what kind of task, what kind of test? Bayesian approaches cover many things, but I don't know what you mean by Bayesian test. Also the only way to know which algorithm performs best is to experiment with your data (en.wikipedia.org/wiki/No_free_lunch_theorem) $\endgroup$ – Erwan Sep 22 '19 at 13:01
  • $\begingroup$ Let me explain a bit. I have to perform regression based prediction on two different algorithms. The output is absolute residual I.e difference between actual and predicted values. For instance, algorithm A has results like 0.3, 0.12, 0.01, 0.7, 0.32 etc and algorithm B has values like 0.1, 0.9, 0.56, 0.32, 0.01 etc. The total instances could be more than 100. Now in weka, I got the results that algorithm A is better than B. Now I want to perform a statistical test that these results are not equivalent. In case of Wilcox test, for instance if I get p value less than 0.05, I say A is sig better $\endgroup$ – Khan Sep 22 '19 at 17:02
  • $\begingroup$ I need the same alternative of Wilcox test but not of frequentist type. It could be a baysian type or other similar type? $\endgroup$ – Khan Sep 22 '19 at 17:04
  • $\begingroup$ Please do not use the comments space for such (long...) clarifications - edit & update your post instead $\endgroup$ – desertnaut Sep 22 '19 at 20:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.