2
$\begingroup$

I am working on Twitter sentiment analysis. Now, I should prepare a training data set that fits for any kind of Twitter data that predicts the sentiment of a tweet as pos, neg, and neu.

From googling, I found an air-line tweets training data set with needed labels (pos, neg, neu, and sentiment). When I checked the sentiment of tweets with that training data set, it gave me mixed results. I found that some positive tweets are given as negative and negative as positive.

Is the method that I am following correct or not?

Please suggest your views for preparing a training data set that can predict any kind of the twitter data.

$\endgroup$
1
$\begingroup$

Since you have trained your model on air-line tweets, the model will learn the characteristics of the air-line tweets. There could be words used in air-line tweets which are positive and contain great weightage but never even used in other tweets leading to negative results. I suggest you to try

  1. Get a dataset which contains all kinds of tweets which you desire essentially similar to what your task ahead is.
  2. Look into Transfer learning.
|improve this answer|||||
$\endgroup$
  • $\begingroup$ okay.so if i want to get the sentiment on the presidential elections of usa then I need to train the model with that kind of the tweets with correct sentiments and feed the model with that kind of the election tweets only ! Right ? $\endgroup$ – Dilip Bobby Jul 6 '16 at 10:37
  • $\begingroup$ Preferably, yes. Then you can get the most accurate results. $\endgroup$ – Hima Varsha Jul 6 '16 at 10:43
  • $\begingroup$ okay!should I maintain same amount of data in my pos,neg,neu training model? or can I go with what I have? I'm have 2k pos,2k neu,7k neg labelled tweets. $\endgroup$ – Dilip Bobby Jul 7 '16 at 5:05
  • $\begingroup$ Need not necessarily. You can try with that and hopefully the model will not overfit for the negative tweets. Give it a shot if you have the data and check the results before taking the next step. $\endgroup$ – Hima Varsha Jul 7 '16 at 5:17
  • $\begingroup$ How to eliminate over fitting problem of a model.Neg set dominates the model :( even if I maintain same size. $\endgroup$ – Dilip Bobby Jul 7 '16 at 5:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.