I'm new to ML and training classifiers in practice, so I was just wondering what the difference was between the built-in sentiment tools of packages such as NLTK and TextBlob as compared to manually creating a classifier (training, testing, etc). I think I read in a comment somewhere that Textblob/NLTK's existing sentiment analysis tools basically just tokenize the text and count the number of positive/negative words to determine an overall sentiment rating (not sure how accurate this is). Does anyone know if using a custom classifier would, in general, be a better way to doing sentiment analysis of text (I'm looking at analyzing the sentiments expressed in hotel reviews)?
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I would say that the meaningful difference in approaches to sentiment classification is between knowledge-based and statistical ones.
The knowledge-based, as you mention, usually use a polarity lexicon, that contains words with a sentiment value and then calculate the sentiment of a text by summing up the values of the words.
The statistical ones train a model based on a labeled training set (i.e. in a supervised setting). The models that are used for that differ, for instance you can use a Naive Bayes classifier or an SVM or any sort of neural network.
With regards to the packages you mentioned, as far as I understand Textblob indeed uses a lexicon. NLTK provides a lexicon-based sentiment classification but it also allows you to train your own statistical model.
If a knowledge-based or a statistical approach is better for you use-case depends really on your data. Same holds for the difference between off-the-shelf vs custom trained one. But it also depends on the time you are able to spend on creating your own model and the expertise you can build on.
If your domain is quite limited, I would argue hotel reviews are quite limited, fine-tuning a knowledge-based approach (by tweaking the underlying lexicon) might give you good results.
In any case, I strongly suggest that you have a test set to evaluate your performance when creating or fine-tuning your model or comparing different models.
$\begingroup$
$\endgroup$
In general, training your own classifier is likely to perform better but it's going to take more time and effort:
- Probably the NLTK system was trained on some generic data which might be very different from your target text. Since any supervised system assumes the same distribution between the training and test set, it's always better to train on a sample of your target data.
- By implementing your own method you can adapt it more specifically to your use case.
So it's a trade-off between the level of quality and the time you're ready to spend on annotating your own data and tuning your own classifier.
