30
votes
Accepted
What is purpose of the [CLS] token and why is its encoding output important?
CLS stands for classification and its there to represent sentence-level classification.
In short in order to make pooling scheme of BERT work this tag was introduced. I suggest reading up on this blog ...
- 5,539
28
votes
Accepted
NLP - why is "not" a stop word?
Stop words are usually thought of as "the most common words in a language". However, other definitions based on different tasks are possible.
It clearly makes sense to consider 'not' as a stop word ...
oW_♦
- 6,185
21
votes
What is purpose of the [CLS] token and why is its encoding output important?
[CLS] stands for classification. It is added at the beginning because the training tasks here is sentence classification. And because they need an input that can represent the meaning of the entire ...
- 311
16
votes
What is purpose of the [CLS] token and why is its encoding output important?
In order to better understand the role of [CLS] let's recall that BERT model has been trained on 2 main tasks:
Masked language modeling: some random words are masked with [MASK]
token, the model ...
- 261
13
votes
What is purpose of the [CLS] token and why is its encoding output important?
Here're my understandings:
(1)[CLS] appears at the very beginning of each sentence, it has a fixed embedding and a fix positional embedding, thus this token contains no information itself.
(2)However, ...
- 131
10
votes
Sentiment retriving from text (Russian)
Take a look at the polyglot library. It has polarity lexicons for 136 languages, including Russian.
The scale of the words’ polarity consisted of three degrees: +1 for
positive words, and -1 for ...
- 101
10
votes
Accepted
Features of word vectors in Word2Vec
1- The number of features: In terms of neural network model it represents the number of neurons in the projection(hidden) layer. As the projection layer is built upon distributional hypothesis, ...
- 1,954
9
votes
Accepted
What is parts of speech technique in sentiment analysis?
Parts of Speech (POS)
This is what it is called when you label each of the words (often called tokens) of a sentence or many sentences. Usually they are labelled with grammatical descriptions, such ...
- 14.5k
8
votes
Training Dataset for Sentiment Analysis of Movie Reviews
You can use the SAR14 dataset of 234K IMDb movie reviews. The construction of the SAR14 dataset is detailed in the paper "Sentiment Classification on Polarity Reviews: An Empirical Study Using Rating-...
- 211
7
votes
How to overcome training example's different lengths when working with Word Embeddings (word2vec)
Let me suggest three simple options:
average the vectors (component-wise), i.e., compute the word embedding vector for each word in the text, and average them. (as suggested by others).
take the (...
- 3,282
7
votes
Improving accuracy of Text Classification
First of all good job done in processing the data and coming up with your base model. I would suggest few things that you can try:
Improve your model my adding bigrams and tri-grams as features.
Try ...
7
votes
Accepted
Why using a frozen embedding layer in an LSTM model
The embedding matrix which used in the initialization of the Embedding layer is highly trained on a large corpus of text. The training and the data are so huge that ...
- 2,085
6
votes
Accepted
Sentiment retriving from text (Russian)
If you are seeking a working solution, I know of an API that supports many languages, including Russian: indico.io Text Analysis sentiment()
...
6
votes
5
votes
How do I assess which sentiment classifier is best for my project?
A couple of important points:
Sentiment analysis is not an exact science. Two people, reading the same text in different contexts will come to different conclusions about sentiment, especially on ...
- 28.3k
5
votes
Accepted
Range to define emotions
The final range of emotion is completely arbitrary. No matter the interval [a, b], you can adjust the emotions to fit inside. [-100, 100] is perfectly reasonable and is common. An example of use is ...
- 206
5
votes
How to add more features in addition to a 100D word vector
You can add a second Input layer to your architecture.
Look at this link Keras Guide Multiple Inputs for a more detailed explanation.
The output of the embedding layer can be combined with the ...
- 753
5
votes
Accepted
How does one go about feature extraction for training labelled tweets for sentiment analysis?
Feature Extraction is an important step when dealing with natural languages because the text you've collected isn't in a form understandable by a computer. If you have a tweet that goes something like
...
5
votes
Accepted
NLP - How to perform semantic analysis?
With your three labels: positive, neutral or negative - it seems you are talking more about sentiment analysis. This answer the question: what are the emotions of the person who wrote this piece of ...
- 14.5k
5
votes
BPE vs WordPiece Tokenization - when to use / which?
(This answer was originally a comment)
You can find the algorithmic difference here. In practical terms, their main difference is that BPE places the @@ at the end ...
- 20.5k
5
votes
Binary classification and numerical labels
If you’re going to have more than two labels, you need to go with a softmax activation and a loss for multi class classification, ie cross entropy loss.
Also, be cautious for multi-class versus multi-...
- 1,948
4
votes
How to overcome training example's different lengths when working with Word Embeddings (word2vec)
Two very different suggestions here to avoid averging the vectors:
Use Word Mover's Distance (https://github.com/mkusner/wmd) to compute distance between the tweets (not sure how well it would work ...
- 49
4
votes
How do I assess which sentiment classifier is best for my project?
You can classify a few of the tweets yourself, and compare afterwards which of the two algorithmic results is closer to your classification.
Without more information we cannot tell what these ...
- 582
4
votes
Sentiment Analysis Tutorial
The NLTK book is by far the best tutorial on basic NLP I have seen(in Python).
The Coursera course on NLP is also fairly good. It takes off from the basics and takes the student to a novice level.
- 8,156
4
votes
Accepted
Understanding of naive bayes: computing the conditional probabilities
Your formula is correct for one $w_i$, but if you want to classify a document, you need to compute $P(c | w_1,\ldots,w_N)$.
Then you have $$P(c | w_1,\ldots,w_N) = \frac{P(c)\cdot P(w_1,\ldots,w_N|c)}...
oW_♦
- 6,185
4
votes
Accepted
Naive Bayes for SA in Scikit Learn - how does it work
Collecting your data
From the comments you state that you wish to classify comments into a label (1-poor, 2-fair, 3-ok, 4-good, 5-very good). Thus you will be training a model that maps a set of ...
- 8,686
4
votes
Accepted
Using Trainable=True in Keras Embedding obtained better performance
It depends upon where do you want to submit your results that you claim, and what is the submission criteria.
First, it is unclear if "lower error in regression" is training or validation/...
- 308
4
votes
How is it possible for RNN to do sentiment analysis?
RNNs do not learn to predict sentiment. They learn correlations between the input data and the target labels. If they see that every time the input contains the word "bad" they have to ...
- 20.5k
3
votes
Sentiment Analysis model for Spanish
The Indico.io API supports Spanish (and
Chinese (Mandarin), Japanese, Italian, French, Russian, Arabic, German, English).
eg in Python:
...
3
votes
Training Dataset for Sentiment Analysis of Movie Reviews
There are many datasets available.
Multi-Domain Sentiment Dataset
Twitter sentiment
UCI
Sentiment Analysis Dataset
Large Movie Review Dataset
- 1,890
Only top scored, non community-wiki answers of a minimum length are eligible