For a 6 class sentence classification task (emotion), I have a list of sentences where I retrieve the sentiment using a language model that was trained on Tweets (bertweet).

It works fine for simplistic sentences where the sentiment is also obvious (someone died, someone won something, someone was afraid of something, etc). However, when applying it to articles, it shows uncontrollable behavior.

Two examples of the sadness class:

How Your Family Can Volunteer During the Pandemic: 99% probability of sadness
There was a massacre in Bosnia where many were slaughtered: 96% probability of sadness

I have tried removing the softmax to break down the probabilities into absolute values in order to see if there's a difference there, but it seems that it is marginal and the first sentence again is "sadder" than the second one about the massacre.

There are many more such examples for all the other classes. Is there any model that is trained on articles? Possibly click bait titles and the kinds?


1 Answer 1


So, I think the interpretation of the model is part of the problem.

For the first: "How Your Family Can Volunteer During the Pandemic" them model is not 99% sad. It is 99% confident based on your training, that the context of this sentence is sad.

The same holds true for the second sentence: "There was a massacre in Bosnia where many were slaughtered" the model is 96% confident that sad is the sentiment of this observation.

Those probabilities are not intensities of the sentiment. So, if you are finding that globally your model is performing poorly on many or most test cases, it suggests one of a few things:

  1. your model is not robust enough or trained on text of different length with different contextually complexities or there is too little training data

  2. your labels are not appropriate to the sentiment you expect. Remember what is considered sad is determined by your input labels.

  3. that massacre and slaughter might not be in the training set, or fully represented or used together such that when combined you get a high probability of sadness

In my experience this happens often when people cherry pick sentences and say, "look how awful this NLP model was on this: X." It happens frequently at my job. But remember, a neural network is designed to emulate a human neural network with appropriate training. Human beings are constantly taking in input and augmenting our training. Subtlety and consistency of our assessment of sentiment is the result of billions of exposures. Your model only has what you have trained it on.

Expecting it to always be right is not reasonable. Also it is not the case that the probability of a sentiment will scale relative to our sense of intensity. The goal of the model is to pick a class. That is done by identifying a sentiment and giving you a probability.

As the creator of the model, you decide where the cutoff value is.

As for the model itself, you are showing what look like titles.

Are you training on article titles or the article itself?

Titles and articles are not the same thing linguistically. Using a model that was designed for tweets to assess long-form copy also might not provide the best generalization capabilities depending on your methods.

In my personal experiences, the labeling itself and the pre-work (wrangling, grooming and such) which you do on language models (including the relative similarity in size and shape of training copy) has a lot more to do with results than small changes in the model structure.

I would start by looking at the global accuracy and then look at all the cases where if fails and see if there is something missing in the training data that you can augment to correct the erroneous predictions.

  • $\begingroup$ Outputs of the model are getting cherry picked here as well, so how do you argue against it? Do you list the training metrics? With regards to the title / content differentiation, the model is trained on millions of tweets, so it has no representation for what a title is, which is the reason behind my question if there are any models trained on articles, or even scientific abstracts and the sorts. What would your suggestion be? $\endgroup$ Commented Nov 10, 2022 at 7:26
  • $\begingroup$ I am not sure how you see the outputs being cherry-picked in my statement above. I said build a model based on an appropriate metric. Optimize for that metric until you cannot anymore. That is not cherry-picking. Cherry picking is training a whole model (maybe you have a metric or not) and then judging its global utility based on one or two poor performing observations. If all of the predictions are bad and those are examples, that is not cherry picking, but if you are evaluating the whole model based on one failure which to a human brain seems obvious, that is cherry-picking $\endgroup$ Commented Nov 14, 2022 at 2:04
  • $\begingroup$ Once you have a solid model (based on objective metrics) you can try to chase down patterns in the wrongness of you model. You may be able to improve the model with more data, some different data or different pre-processing. $\endgroup$ Commented Nov 14, 2022 at 2:05
  • 1
    $\begingroup$ Ah, I get it. Yes, that happens to me too. I hired 4 interns to label 100k observations, and do a comprehensive accuracy measure for our NLP model which has about 2000 target topics. It was 75% accurate (and the context for identifying a question to answer in our space is hard) which is actually good in our space and still I get "Look at this one question we got wrong, this is ridiculous" It is so frustrating because we have 10-15 complaints out of 4-million incoming messages and people say our AI sucks. $\endgroup$ Commented Nov 14, 2022 at 16:03
  • 1
    $\begingroup$ Our solution is to build a training program company-wide for anyone who contacts our customers that explains what AI is and is not, when it succeeds and when it fails and why you cannot look at one prediction to define the utility of a whole ai system. $\endgroup$ Commented Nov 14, 2022 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.