As a part of my university project, I am researching/developing a sentiment analysis model where in I am trying to combine NLTK Vader (SentimentIntensityAnalyzer) results with a Machine Learning trained classifier for prediction of Sentiments on Twitter data.

Detailed description -

To explain what I am trying to do is - Combine Machine Learning classifier and NLTK Vader sentiment analysis to get better classification of tweets as Positive, Negative or Neutral.

What I have done is -

  1. Cleaned the data (Niek Sanders twitter corpus) and pre-processed the tweets which includes stop words removal, URL removal, User mention removal, remove # symbol from Hashtags, lower case conversion, stemmer processing, etc.

  2. Split into 80:20 training:test ratio

  3. Used TfidfVectorizer to create sparse matrix of features with TFIDF of words. The number of columns is equal to the number of words in clean data.

  4. Use this vector matrix to train and test the classifiers using scikit-learn.

Classifiers used - This same vector matrix is being used to train - KNN, Random forest, Naive Bayes, SVM, Artificial Neural Network and Convolutional Neural Network.

Now the main doubt arises when trying to combine NLTK Vader (SentimentIntensityAnalyzer results).

What I am doing is - From step three above, in the tfidf vector matrix I am adding 2 columns and I am adding the Positive and Negative polarity result for the tweet given by NLTK, so now the vector matrix has 2 new columns n+1 (total positive polarity of tweet by NLTK), n+2 (total negative polarity of tweet by NLTK) and it looks like -

    0      1     2     3     4     5     6     7  ......... |  n+1    |   n+2
------------------------------------------------------------|---------|----------
    0.4   0.3   0.4   0.1   0.5   0.3   0.2   0.4 ......... |  0.345  |  0.345
    0.5   0.3   0.2   0.8   0.3   0.6   0.4   0.5 ......... |  0.765  |  0.523
    0.6   0.4   0.1   0.7   0.8   0.8   0.2   0.2 ......... |  0.392  |  0.664
    0.2   0.9   0.7   0.4   0.9   0.9   0.8   0.5 ......... |  0.832  |  0.658
    0.9   0.5   0.9   0.7   0.3   0.2   0.2   0.5 ......... |  0.273  |  0.283
    0.5   0.2   0.2   0.7   0.2   0.1   0.6   0.6 ......... |  0.505  |  0.194
    0.4   0.3   0.2   0.3   0.3   0.9   0.5   0.5 ......... |  0.102  |  0.927
    0.1   0.8   0.1   0.2   0.1   0.5   0.2   0.7 ......... |  0.735  |  0.455

Question - So is it correct to add it this way?

I could also convert these polarities to binary values to mark if the overall sentiment is positive, negative or neutral using 0 and 1.

Note - I am representing 3 categories(positive, negative or neutral) using 2 columns to avoid dummy variable trap.

Though it has increased the overall accuracy of the classifiers by small margin of 1-2%.

But am I doing it right? If not, kindly let me know how can I collaborate the two mentioned above, i.e. NLTK Vader results with Machine Learning classifiers.

  • I'm answering you a year later, I hope this is still relevant to you. So, I came across your post as I'm facing the same problem at the moment. I'm researching on sentiment analysis for social media in Chinese. For my first baseline, I made my own implementation of VADER for Chinese with the goal to predict sentiment for Weibo. With some modifications it works reasonably well ~ 90% accuracy. I introduced some POS rules to make sentiment predictions, but unfortunately, Chinese language is a bit more complicated than English and there are not much good lexicons available. Therefore, I wish to im – Thiago Aug 10 at 9:27

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.