1
$\begingroup$

I am doing classification by using bag-of-words model. The goal is to locate users based on their tweets. Splitted the data as 80% training and 20% test.

I did experiments with sklearn's SVC and Naives Bayes. The results 35% and 42% accuracy respectively. However, when I try the sklearn's LinearSVC algorithm, it gives me 80% which is shocking.

This is the part of the code:

text_clf = Pipeline([
    ('vect', CountVectorizer(stop_words='english')),
    ('tfidf', TfidfTransformer()),
    ('clf', LinearSVC()),
])
text_clf.fit(train_data, train_target)  

What might be the reason for that? Why LinearSVC performs really good?

$\endgroup$

1 Answer 1

2
$\begingroup$

These models are used for convex optimization. which means that there is only one solution for the problem. LinearSVC and SVC are used for same purpose but the optimization techniques used are different. E.g. LinearSVC intercept is penalized while in SVC it isn't. there might be differences scaling or default loss function. Hence they produce different results. LinearSVC tends to optimize faster. in multiclass classification, liblinear does one-vs-rest by default whereas libsvm does one-vs-one.

if you want to know how they can produce similar results. You can check out this question. https://stackoverflow.com/q/33843981/5947203

Now the reason why SVN produced better results than Naive Bayes is that in this problem features are important. Naive Bayes treat them as independent while SVN looks at the interactions between them to a certain degree. Mathematically, One is probabilistic while other is geometrical. The dependencies are not captured by Naive Bayes hence it does not produce good results.

$\endgroup$
2
  • $\begingroup$ Overfitting is possible for linearsvc? Because the difference between the SVC and LinearSVC is double. $\endgroup$
    – Mert Metin
    Feb 14, 2019 at 22:30
  • $\begingroup$ Most of the SVCs do not overfit. You can refer to this question for more details -> stats.stackexchange.com/questions/35276/… $\endgroup$
    – Ani
    Oct 31, 2020 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.