3
$\begingroup$

Okay, here is the background: I am doing text mining, and my basic flow is like this: extract feature (n-gram), reduce feature count, score (tf-idf) and classify. for my own sake i am doing comparison between SVM and neural network classifiers. here is the weird part (or am i wrong and this is reasonable?), if i use 2gram the classifiers' result (accuracy/precision) is different and the SVM is the better one; but when i use 3-gram the results are exactly the same. what causes this? is there any explanation? is it the case of very separable classes?

$\endgroup$
5
  • $\begingroup$ Do the test set predictions for the two classifiers correspond exactly? i.e. is every instance classified in the same way? $\endgroup$
    – Ansari
    Oct 5, 2014 at 1:30
  • $\begingroup$ yes the methods are exactly like each other. the only difference is from the feature type $\endgroup$ Oct 5, 2014 at 5:47
  • $\begingroup$ Not the methods, I meant the classified labels. Do they correspond exactly or is it just the aggregate percentages that match? $\endgroup$
    – Ansari
    Oct 6, 2014 at 4:06
  • $\begingroup$ Aaah. I misunderstood, sorry. but it makes no difference, they are the same $\endgroup$ Oct 6, 2014 at 21:21
  • $\begingroup$ I asked a similar question not too long ago: datascience.stackexchange.com/questions/992/… $\endgroup$
    – JenSCDC
    Oct 7, 2014 at 1:49

1 Answer 1

0
$\begingroup$

Your results are reasonable. Your data brings several ideas to mind:

1) It is quite reasonable that as you change the available features, this will change the relative performance of machine learning methods. This happens quite a lot. Which machine learning method performs best often depends on the features, so as you change the features the best method changes.

2) It is reasonable that in some cases, disparate models will reach the exact same results. This is most likely in the case where the number of data points is low enough or the data is separable enough that both models reach the exact same conclusions for all test points.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.