6
$\begingroup$

I'm wondering how RNN can be used when doing sentiment analysis.

It seems that the characteristic of RNN is to remember what appeared in the past and determine the value of the present (future), but I think this RNN's characteristic is useless when analyzing sentiment.

I am curious to see how it is possible to do emotional analysis by remembering the words that have appeared so far.

I think the way to train vectors that show how many times specific words appear is the only way to analyze sentiment.

If some RNN model train a sentence "good movie" and label "positive", can it predict a new sentence "movie good" as a "positive"?

Also, if a normal model(just fully connected deep neural network) train a sentence "good movie" and label "positive", is it hard for this model to predict a new sentence "movie good" as a "positive"?

Improve this question
$\endgroup$
1
  • $\begingroup$ Please, consider marking some of the answers as correct if deemed so. Alternatively, please considering describing what the answers are lacking or why you think they are not correct, so that they can be improved. $\endgroup$
    – noe
    Dec 10, 2020 at 10:45

2 Answers 2

Reset to default
4
$\begingroup$

RNNs do not learn to predict sentiment. They learn correlations between the input data and the target labels. If they see that every time the input contains the word "bad" they have to generate the label "negative", then they will learn it. If they see in the training data that the previous phenomenon happens always except when there is a "not" before "bad", then they will learn it. Depending on how the data distribution is, they may generalize the negation to any combination of verbs and adjectives, or maybe not, and they only handle negation appropriately when the input data is very very similar to the training data.

Your question seems to aim at understanding the "internal dynamics" of the RNN when it is predicting sentiment. While there is some research in that direction, I think they can still be considered as "black boxes", in that we do not actually understand the functions modeled by neural networks and therefore their outputs are not "explainable"

Improve this answer
$\endgroup$
1
$\begingroup$

It depends on how you configure your model and the layers you're using. You can add other layer's of DNN and experiment with those.

You should also consider using LSTM(Long short-term memory), since the past information do not get decayed quickly like in RNN, and you keep the present values, thus having a better model based on RNN.

You can also you a Bidirectional LSTM (BLSTM), which improve the learning process by using a cell state that runs through all your past data and generalize well for sentiment analysis.

Keep in mind that RNN cannot detect negation in sentences, like saying "This movie was not good", usually ANN need more configuration to understand those kind of aspects.

Another trick is to add a CNN layer to your model, so it can extract key words, thus when your data is forwarded to the RNN based layers, it will less noisy and more oriented to specific keywords.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Why do you think that "RNN cannot detect negation in sentences"? $\endgroup$
    – noe
    Oct 26, 2020 at 10:44
  • $\begingroup$ @ncasas in NLP some concepts involving irony, sarcams, negations, jokes to name a few needs a lot of context information, and coding them is not straightforward. For example if you have the word "good" in sentence your model will likely to determine it as positive, and for the example "not as good as", what do you think you're model will classify it as? so you need to explicitly code some negation terms yourself, then even with a simple RNN you'll have a better reults. $\endgroup$
    – MXK
    Oct 26, 2020 at 12:00
  • $\begingroup$ @ncasas I agree with you that the ANN's are like black boxes, but in you choose a fairly simple dataset, then have a few layers and neurons, this way you can track down the ANN's behavior. $\endgroup$
    – MXK
    Oct 26, 2020 at 12:15
  • $\begingroup$ I don't think any natural language dataset can be simple enough to "track" it. Of course, you can measure how the model behaves under certain input patterns, but that won't enable you to reason about the model's behavior on unseen data, so any explanation you derive from the "tracking" would probably just be post hoc . $\endgroup$
    – noe
    Oct 26, 2020 at 12:44
  • 1
    $\begingroup$ And the statements RNN cannot detect negation in sentences and in NLP some concepts involving irony, sarcams, negations, jokes to name a few needs a lot of context information, and coding them is not straightforward are very different. I agree with the second one and strongly disagree on the first one. $\endgroup$
    – noe
    Oct 26, 2020 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.