0
$\begingroup$

I have a binary feature that i want to use it with textual features i.e. unigrams. I use logistic regression and TF/IDF for representing text. So i simply add a unique feature, say ss or oo, to text of each instances. But in practice, i see adding more number of these features to instances, say two oo or ss or more get me a better results. What is the reason? How these weights improve the classification results? Should not logistic regression can get more weights to this features instead of weighting them by hand?

$\endgroup$
4
  • 1
    $\begingroup$ Do you add a feature symbol to the text? Why don't you add the feature after the text processing (e.g., after you computed tf/idf)? $\endgroup$
    – DaL
    Jun 14 '17 at 5:48
  • $\begingroup$ yes, i add a feature symbol to text of each comment. I also in a second way the feature to sparse matrix of tf/idf using hstack of scikit learn.....But my primary question now is that, when using repeat of a feature and increasing its frequency it is possible to improve the model, why logistic regression itself can't learn this weight in case of adding only one symbol feature? $\endgroup$
    – keramat
    Jun 14 '17 at 11:25
  • $\begingroup$ for example: text of first instance + ss, text of first instance ss ss ss ss ss. In this example i get more accuracy in later. $\endgroup$
    – keramat
    Jun 14 '17 at 11:27
  • 1
    $\begingroup$ By adding the symbol to the text, you mix it with the tex in ways that might be not intentional. For a short text the idf will be higher than in a long text. adding the symbol few times might help coping with this problem. However, the problem doesn't appear in the first place if instead of adding the feature to the text you'll add the feature after processing the text. $\endgroup$
    – DaL
    Jun 15 '17 at 11:00
-1
$\begingroup$

hstak is a good solution. Using this I combined these features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.