Okay, here is the background: I am doing text mining, and my basic flow is like this: extract feature (n-gram), reduce feature count, score (tf-idf) and classify. for my own sake i am doing comparison between SVM and neural network classifiers. here is the weird part (or am i wrong and this is reasonable?), if i use 2gram the classifiers' result (accuracy/precision) is different and the SVM is the better one; but when i use 3-gram the results are exactly the same. what causes this? is there any explanation? is it the case of very separable classes?

  • $\begingroup$ Do the test set predictions for the two classifiers correspond exactly? i.e. is every instance classified in the same way? $\endgroup$ – Ansari Oct 5 '14 at 1:30
  • $\begingroup$ yes the methods are exactly like each other. the only difference is from the feature type $\endgroup$ – Nima Shayanfar Oct 5 '14 at 5:47
  • $\begingroup$ Not the methods, I meant the classified labels. Do they correspond exactly or is it just the aggregate percentages that match? $\endgroup$ – Ansari Oct 6 '14 at 4:06
  • $\begingroup$ Aaah. I misunderstood, sorry. but it makes no difference, they are the same $\endgroup$ – Nima Shayanfar Oct 6 '14 at 21:21
  • $\begingroup$ I asked a similar question not too long ago: datascience.stackexchange.com/questions/992/… $\endgroup$ – JenSCDC Oct 7 '14 at 1:49

Your results are reasonable. Your data brings several ideas to mind:

1) It is quite reasonable that as you change the available features, this will change the relative performance of machine learning methods. This happens quite a lot. Which machine learning method performs best often depends on the features, so as you change the features the best method changes.

2) It is reasonable that in some cases, disparate models will reach the exact same results. This is most likely in the case where the number of data points is low enough or the data is separable enough that both models reach the exact same conclusions for all test points.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.