How to implement your own word list for sentiment analysis?

Question

I've finished web scraping and cleaning the text i'm interested in and i also have the sentiment word list i want to apply to it. However, i have a few conceptual and implementation problems.

my sentiment word list is a data frame of words assigned to particular features such as negative, positive etc. I currently have a list where each element is a block of text waiting to be analysed. I'm not sure where to go from here. From the resources i have seen so far, they put their text through a sentiment word list from another module that takes care of it for you and simply gives you the results.

I need clarification on the following:

1. my sentiment word list contains words and they are either True(1)/False(0) on specific features such as positive and negative. So within my text thats being analysed, if it contains the words that are also in my sentiment word list, they will be added to the document matrix, along with their frequency. The overall positivity and negativity will be proportional the the frequency of positive and negative words. Have i understood this correctly?

2. How do i go about implementing this? i'm not sure on how to use the vocabulary parameter within Count Vectorizer.

from sklearn.feature_extraction.text import CountVectorizer
vc=CountVectorizer(vocabulary=sentiment_words)
vectors=vc.fit_transform(cleaned_text)

The feature names end up being the features i'm using as an independent variable.

(vc.get_feature_names())
['word',
 'negative',
 'positive',
 'uncertainty',]

The feature names should be the actual words contained within my sentiment word list right? i'm assuming i need to change the sentiment word list dataframe but i'm not sure on how to proceed. Also, how would the overall positive and negative metric be assigned here?

I'm quite new to this so any sort of clarification would be great! Thanks!

Answer 1

A relatively straightforward approach is counting up the number of occurrences of positive and negative words. Then see which tally is larger.

Creating a sentiment word list dataframe is not the best approach. Dataframe are large in memory and strings are large in memory. It is better to create a hash table, aka Python dict, to reduce the data size in memory.

The most common options for assigning positive and negative metric are:

1. Use an existing metric, such as TextBlob's sentiment.
2. Assign your own metric score.