# Tag Info

26

Stop words are usually thought of as "the most common words in a language". However, other definitions based on different tasks are possible. It clearly makes sense to consider 'not' as a stop word if your task is based on word frequencies (e.g. tf–idf analysis for document classification). If you're concerned with the context (e.g. sentiment analysis) ...

14

CLS stands for classification and its there to represent sentence-level classification. In short in order to make pooling scheme of BERT work this tag was introduced. I suggest reading up on this blog where this is also covered in detail.

11

[CLS] stands for classification. It is added at the beginning because the training tasks here is sentence classification. And because they need an input that can represent the meaning of the entire sentence, they introduce a new tag. They can’t take any other word from the input sequence, because the output of that is the word representation. So they add a ...

10

Most often serialization error in (Py)Spark means that some part of your distributed code (e.g. functions passed to map) has dependencies on non-serializable data. Consider following example: rdd = sc.parallelize(range(5)) rdd = rdd.map(lambda x: x + 1) rdd.collect() Here you have distributed collection and lambda function to send to all workers. Lambda ...

10

Take a look at the polyglot library. It has polarity lexicons for 136 languages, including Russian. The scale of the words’ polarity consisted of three degrees: +1 for positive words, and -1 for negatives words. Neutral words will have a score of 0. You can use it like this: >>> from polyglot.text import Text as T >>> text = T("это ...

10

1- The number of features: In terms of neural network model it represents the number of neurons in the projection(hidden) layer. As the projection layer is built upon distributional hypothesis, numerical vector for each word signifies it's relation with its context words. These features are learnt by the neural network as this is unsupervised method. Each ...

9

Parts of Speech (POS) This is what it is called when you label each of the words (often called tokens) of a sentence or many sentences. Usually they are labelled with grammatical descriptions, such as Noun, Adjective, Adverb. They can often get quite specific, also distinguishing e.g. between types of nouns (proper nouns etc). You can then use these ...

8

You can use the SAR14 dataset of 234K IMDb movie reviews. The construction of the SAR14 dataset is detailed in the paper "Sentiment Classification on Polarity Reviews: An Empirical Study Using Rating-based Features".

7

First of all good job done in processing the data and coming up with your base model. I would suggest few things that you can try: Improve your model my adding bigrams and tri-grams as features. Try doing some topic modelling like latent Dirichlet allocation or Probabilistic latent Semantic Analysis for the corpus using a specified number of topics - say ...

6

If you are seeking a working solution, I know of an API that supports many languages, including Russian: indico.io Text Analysis sentiment() >>> import indicoio >>> indicoio.config.api_key = YOUR_API_KEY >>> indicoio.sentiment(u"Это круто, убивает! Хочу.", language='ru') 0.6978093435482927 >>> indicoio.sentiment(u"Ты кто ...

6

Let me suggest three simple options: average the vectors (component-wise), i.e., compute the word embedding vector for each word in the text, and average them. (as suggested by others). take the (component-wise) maximum of the vectors. (max, instead of average) take the (component-wise) minimum of the vectors. (min, instead of average) Each of these ...

6

The embedding matrix which used in the initialization of the Embedding layer is highly trained on a large corpus of text. The training and the data are so huge that the embedding has learnt a type of association between words. A pretrained embedding like Word2Vec will produce vectors for words like school and homework which are similar to each other in ...

5

Consider the following two sentences: My awesome girlfriend bought me a delicious popsicle at the store. 0--1-------0----------0------0--0-1---------0--------0--0---0-----:2:11 My awesome girlfriend, Joyce, drove to the grocery store to buy me a delicious Dole popsicle. 0---1------0-----------0------0-----0--0---0-------0-----0--0---0--0-1---------0----0---...

5

A couple of important points: Sentiment analysis is not an exact science. Two people, reading the same text in different contexts will come to different conclusions about sentiment, especially on borderline cases. Perhaps text has complex grammar, or has a metaphor or simile in it where it helps to understand what is actually being compared. The ground ...

5

The final range of emotion is completely arbitrary. No matter the interval [a, b], you can adjust the emotions to fit inside. [-100, 100] is perfectly reasonable and is common. An example of use is from GDELT, which provides this interval for average tone of news documents. Asking if equally distancing the emotions is statistically correct does not make ...

5

You can add a second Input layer to your architecture. Look at this link Keras Guide Multiple Inputs for a more detailed explanation. The output of the embedding layer can be combined with the secondary input and passed to the next layer. auxiliary_input = Input(shape=(5,), name='aux_input') x = keras.layers.concatenate([lstm_out, auxiliary_input])

5

Feature Extraction is an important step when dealing with natural languages because the text you've collected isn't in a form understandable by a computer. If you have a tweet that goes something like I do not like the views of @Candidate1 on #Topic1. Too conservative!! I can't stand it! then we can't just feed this into a learning algorithm. We need to ...

5

With your three labels: positive, neutral or negative - it seems you are talking more about sentiment analysis. This answer the question: what are the emotions of the person who wrote this piece of text? Semantic analysis is a larger term, meaning to analyse the meaning contained within text, not just the sentiment. It looks for relationships among the ...

5

(This answer was originally a comment) You can find the algorithmic difference here. In practical terms, their main difference is that BPE places the @@ at the end of tokens while wordpieces place the ## at the beginning. The main performance difference usually comes not from the algorithm, but the specific implementation, e.g. sentencepiece offers a very ...

5

If you’re going to have more than two labels, you need to go with a softmax activation and a loss for multi class classification, ie cross entropy loss. Also, be cautious for multi-class versus multi-label (below). Multi-class One-of-many classification. Each sample can belong to ONE of $C$ classes. The model will have $C$ output neurons that can be ...

4

You can classify a few of the tweets yourself, and compare afterwards which of the two algorithmic results is closer to your classification. Without more information we cannot tell what these algorithms were doing. It may well be that they were just using different thresholds internally: Algo 1 decided that everything > 60% threshold is "positive", all < ...

4

The NLTK book is by far the best tutorial on basic NLP I have seen(in Python). The Coursera course on NLP is also fairly good. It takes off from the basics and takes the student to a novice level.

4

Your formula is correct for one $w_i$, but if you want to classify a document, you need to compute $P(c | w_1,\ldots,w_N)$. Then you have $$P(c | w_1,\ldots,w_N) = \frac{P(c)\cdot P(w_1,\ldots,w_N|c)}{P(w_1,\ldots,w_N)} = \frac{P(c) \cdot \prod_{i=1}^N P(w_i|c)}{P(w_1,\ldots,w_N)} \neq \prod_{i=1}^NP(c|w_i)$$ where the second equation holds because of the ...

4

Collecting your data From the comments you state that you wish to classify comments into a label (1-poor, 2-fair, 3-ok, 4-good, 5-very good). Thus you will be training a model that maps a set of words (the paragraph) to a number which represents the rating. I will assume that you also have labels for some of the comments which we will call your training set....

4

In order to better understand the role of [CLS] let's recall that BERT model has been trained on 2 main tasks: Masked language modeling: some random words are masked with [MASK] token, the model learns to predict those words during training. For that task we need the [MASK] token. Next sentence prediction: given 2 sentences, the model learns to predict if ...

4

RNNs do not learn to predict sentiment. They learn correlations between the input data and the target labels. If they see that every time the input contains the word "bad" they have to generate the label "negative", then they will learn it. If they see in the training data that the previous phenomenon happens always except when there is a ...

3

There are many datasets available. Multi-Domain Sentiment Dataset Twitter sentiment UCI Sentiment Analysis Dataset Large Movie Review Dataset

3

The quick (and not very satisfying) answer is "it depends" -- specifically it depends upon what your underlying conceptual model of human emotion is and how it manifests in verbal/written behavior. What is your characterization of neutrality in relation to positive and negative valence? Can documents be put on some sort of quantitative scale with neutral ...

3

I was thinking if Apriori might be more suited for your purpose. For your consideration: 1) Tokenize the training sentences into bag of words: Review Sentence | Upside | Calm | Swimming | Cat 2) Tag the correct consequents for bag of words. 3) Apriori should produce 1 rule for ambiance and 1 rule for Entertainment. Hope this helps.

3

The Indico.io API supports Spanish (and Chinese (Mandarin), Japanese, Italian, French, Russian, Arabic, German, English). eg in Python: >>> import indicoio >>> indicoio.config.api_key = <YOUR_API_KEY> >>> indicoio.sentiment("¡Jamás voy a usar esta maldita aplicación! No funciona para nada.") 0.02919392219306888 >>>...

Only top voted, non community-wiki answers of a minimum length are eligible