1
$\begingroup$

Let's imagine I have a dataset of movie reviews with annotated sentiment:

-1 means negative
 0 means neutral
+1 means positive

I see a lot of people trying to do classification to try to answer those types of problems, but shouldn't regression be used instead? To me using regression would allow the system to model that there is a transition between labels, e.g. 0 is in between. Any thoughts on this?

$\endgroup$
1
  • $\begingroup$ Half way between positive and negative (two classes) is not neutral? The reason this option might not be presented to users, if we're talking about a supervised situation, is that they might select it by default, while leaving it out forces them to actively pick. $\endgroup$
    – Emre
    Oct 20 '16 at 18:03
1
$\begingroup$

This is Ordinal Regression https://en.wikipedia.org/wiki/Ordinal_regression

Quote from Wikipedia:

In statistics, ordinal regression (also called "ordinal classification") is a type of regression analysis used for predicting an ordinal variable, i.e. a variable whose value exists on an arbitrary scale where only the relative ordering between different values is significant.

Examples are the ranking system you describe or any question with categorical but ordered answers often seen in surveys ("always", "sometimes", "never").

$\endgroup$
3
$\begingroup$

An issue with regression is that the distance between negative, neutral and positive are not necessarily the same. This transition/uncertainty you mention can be modeled by using a probability distribution over the classes as opposed to a hard argmax. If you use Neural Networks for example the last softmax layer gives you this information for free (For example: 0.1 on Negative, 0.6 Neutral and 0.3 Positive).

$\endgroup$
3
  • $\begingroup$ "If you use Neural Networks for example the last softmax layer gives you this information for free (For example: 0.1 on Negative, 0.6 Neutral and 0.3 Positive)." hum, were you trying to say more something like trying to align to [positive:0.5, negative:0.5] for the neutral instance? $\endgroup$ Oct 20 '16 at 9:52
  • $\begingroup$ Well, if you use 2 classes then yes, but if you add Neutral as a class my example stands. The more it goes to 0.3333 for each the less certain it is about it. $\endgroup$ Oct 20 '16 at 9:54
  • $\begingroup$ Ok, I guess it is possible for a neural net to learn this, but wouldn't it be easier for him if he could find latent patterns in data that just represent how positive/negative the datapoint is (logistic regression style) . With three dimensions the algorithm will have to find discriminative patterns for each class, which is counter-intuitive to me because the neutral class is a lack of positive/negative patterns. I guess the question I am addressing is about handling multiclass problems when classes are not linearly independent. $\endgroup$ Oct 20 '16 at 10:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.