New answers tagged sentiment-analysis
0
The baseline is not the word list by itself, the idea is to implement a simple classifier which works as follows:
It receives as input a list of standard positive words P (e.g. "good", "great", "nice", ...) and a list of standard negative words N (e.g. "bad", "depressing", "annoying",...).
For ...
0
Features of your naive bayes classifier is most likely a bag of words where each feature represents a single word.
The naive bayes classifier also assumes that each feature (or word) contributes independently to the final prediction.
So if your sentence is composed of a single word - you get an idea of how the model reacts to that word.
0
You need to do pip install sentencepiece for it to work.
By the way, you can also give the tweets as a list to the tokenizer. You don't need to tokenize them one by one.
tokenizer(tweets, max_length=max_len, padding='max_length', add_special_tokens=True)
0
(This answer was originally a comment)
You can find the algorithmic difference here. In practical terms, their main difference is that BPE places the @@ at the end of tokens while wordpieces place the ## at the beginning. The main performance difference usually comes not from the algorithm, but the specific implementation, e.g. sentencepiece offers a very ...
Top 50 recent answers are included
