1. I'm trying to create a Sentiment Analysis algorithm for a custom data (government dept specifc data) and not like any other social media data etc. The data exists but I need to categorise the data as positive or negative.
  2. My requirement is to classify the test data as positive or negative using a sentiment analysis algorithm.
  3. The big challenge for me is to take the similar set of government data which I can get. But I need to prepare the training data set. For this I need to classify the training data set as positive or negative.
  4. On what basis do I need to categorise the data set as positive or negative. The reason I'm asking this question is that preparing a good training data set is very important to improve the accuracy of my sentiment analysis algorithm .
  5. I have come up with my sentiment analysis algorithm but really need to prepare a robust set of training data(classified as positive or negative).

Experts - I'm a newbie to the machine learning area and I need advice from the researchers and experts out there.

  • 1
    $\begingroup$ Welcome to Data science SO! Please read the guidelines on posting high quality questions. When posting a question it is common practice to post an approach that you tried. It looks to me that you didn't even write a bit of code to try things out. This is not a place where people are going to provide you with code and sniplets. $\endgroup$
    – Stereo
    Oct 28, 2016 at 8:03
  • $\begingroup$ @stereo, the question is not about the code. I have written the code for performing a sentiment analysis using naive bayes algorithm. The question I'm asking here is that what are the various approaches I need to take while preparing the training data. I have the training data but how do categorize the training data as positive or negative. $\endgroup$
    – Avot
    Oct 30, 2016 at 22:55
  • $\begingroup$ Can you describe your data? What does positive and negative data look like? As is, it is impossible to answer. $\endgroup$
    – Hobbes
    Oct 31, 2016 at 13:03
  • $\begingroup$ You can do word embedding for your corpus and then use a window of 5 words with a center word to classify positive vs negative sentences for example $\endgroup$
    – DavidOooO
    Jan 1, 2017 at 7:41
  • $\begingroup$ you are saying that you need to classify the data you have into positive or negative, but that depends completely on which context you are going to conclude positive on. Each case it changes i.e, positive towards particular behavior or particular state. $\endgroup$ Sep 5, 2017 at 8:38

2 Answers 2


The only way to obtain a high-quality dataset in your specific domain is to do it manually. There exists no other method that can give you the sentiment labels for texts in arbitrary domains. If there would exists such a method, why would you even bother to create your own model.

You should probably find/hire people that can do this work for you. Adding the sentiment meta-data to the texts could be as easy as creating an additional column in excel. But you could also smoothen the process by creating a little app that shows the sentence and let the user decide what sentiment it is (e.g. by swiping to left or right). Find a bunch of people that are willing to classify 10 sentences a day and your dataset will grow steadily. Even better, apply a little gamification and give people points or a rank and give an award to the best performing user to boost their performance.

Keep in mind that if these texts are really domain-specific you probably need domain-experts to do the classification.

When having enough training data you could learn a (simple) that predicts the output. Then you could let the human classifiers focus on the sentences with low confidence, assuming that high confidence automatic classifications are correct.


You can use SentiWordNet for classifying your data. SentiWordNet assigns to each synset of WordNet three sentiment scores: positivity, negativity, objectivity.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.