Why did Logistic regression perform better than svm? [closed]

Question

I have a data set of movies and their subtitles.My task is to classify them based on their ratings-[R,NR,PG,PG-13,G]. I have tried different ML algorithms and found that Logistic regression out performed them all, but I am unable to figure out why.My data had more features than observations.

SVM- should perform well on high dimensional data and will perform well even the there is class imbalance, but failed to show great results. Naive Bayes-I think Naive Bayes did not perform well because of class imbalance. Random forest-decent performance.but did not out perform logistic regression.

I am looking for an explanation as to why did one perform better than the other.

Note:The data set is sparse and it has more features/parameters than observations/examples.

Answer 1

There is a theorem in machine learning literature which is called “No free lunch theorem”. The essence of NFL is that there is no universal model which performs best for every problem and every dataset. So, according to NFL, you can not expect SVM to outperferm logistic regression in all situations and contexts. If your classes were linearly separable SVM would be perfect with 100% accuracy but otherwise you shouldn’t expect it to necessarily outperform ligistic regression. So, whether SVM or ligistic regression are better choice highly depends on the problem and on the available dataset.

Answer 2

As a practical guideline: Linear classifiers (such as LR) can perform very well on extremely sparse datasets.

Answer 3

My guess is that since you have more features than obs, there are more chances that there exists an affine subspace which separates each class from the rest. i.e. Your data becomes linearly separable. Since log. regression has a linear boundary, should be easy to find it.

I have no explanation for the bad performance of others. Are you using a linear kernel for SVM? If not, try it.