2
$\begingroup$

I'm doing some classification with a text corpus [professional emails], I've already done all the basic pre processing steps (stemming, remove stop words, punctuation, low frequency words, words length ...) which gives me a F1-score = 0.6.

I was wondering what could be the next step(s) to improve my results, I thought of "stacking" (use multiple classifiers) and maybe removing words which are common to the different classes (4 differents types of texts) [ not sure about this solution ]

Edit:

I tried with 15 differents classifier the best one is gradient boosting (gbm )with an F1-score=0.6002159. The features have been extracted through a bag of words ( Document Term Matrix ) with Tf-Idf [I don't think taking into account the length of a text, punctuation or others is relevant] - [It is mainly professional emails]

Improve this question
$\endgroup$
4
  • $\begingroup$ Can you provide more details - how do you extract features from the text, what classifier are you using,..? $\endgroup$
    – stmax
    Commented Jun 28, 2016 at 13:51
  • $\begingroup$ F1 score punishes false negatives and false positives equally. So, if your classes are not uniformly distributed, you should train with equalizing weights. In sklearn, this would mean using class_weight=balanced for whatever model you are using. $\endgroup$
    – Ricardo Cruz
    Commented Jun 30, 2016 at 16:53
  • $\begingroup$ I am curious about what are the labels of the classification. More problem specific solution might be proposed if the problem is fully described. $\endgroup$
    – William
    Commented Jul 1, 2016 at 19:46
  • $\begingroup$ The labels are basically two things: "Requests" and "Incident" about some softwares $\endgroup$
    – bouritosse
    Commented Jul 4, 2016 at 8:00

2 Answers 2

Reset to default
3
$\begingroup$

Have you tried without the Tf-Idf weighting? What about a bi-gram analysis?

You can also review the pre processing, maybe without removing the low frequency words (wich the tf idf already does). Another idea is to remove one very high frequency word that appear on your corpora.

Improve this answer
$\endgroup$
1
  • $\begingroup$ No didn't try without Tf-idf or bi-gram, I'm gonna try thanks. And I'm gonna remove less low frequency words that's a good idea as well $\endgroup$
    – bouritosse
    Commented Jul 4, 2016 at 7:58
2
$\begingroup$

Feature engineering is usually the way to improve the performance.

Another possible way to improve result is through supervised feature selection. I could recommend Chi-square feature selection, step-wise feature selection. As for a more complete survey on feature selection, you could refer to Guyon, Isabelle, and André Elisseeff. "An introduction to variable and feature selection." Journal of machine learning research 3.Mar (2003): 1157-1182.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I don't know if it is really necessary I'm using the caret package and if I'm not wrong the feature selection is already done by the Train() function $\endgroup$
    – bouritosse
    Commented Jul 4, 2016 at 7:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.