I have a dataset of posts for sentiment analysis that are labelled with -1 (negative), 1 (positive) or 0 (neutral).

So I wonder how should I deal with that. These are my ideas:

  • make a multiclass classifier : I tried with a random forest, and the results are pretty correct; however, I have a certain amount of negatives in positives and vice versa; I would've preferred the error to rather be in neutral.
  • make a binary classifier, but when predicting, if the probabilities are too balanced, return neutral. However, it seems to me that I don't use the neutral data - isn't it a waste of data? Maybe using a OneVsAll will be better?
  • make a perceptron/neural network with an tanh neuron at the output; but I don't know what good loss function could be used here.

Do you know if there is any of those that is theoretically/practically better?

  • 1
    $\begingroup$ refer here - medium.com/@mattkiser/… $\endgroup$
    – Academic
    Commented Nov 3, 2020 at 16:15
  • $\begingroup$ For your third point, loss function depends on your classification type. If binary then go for binary cross entropy loss and if multiclass use sparse categorical cross entropy loss. $\endgroup$
    – spectre
    Commented Dec 14, 2021 at 3:36

1 Answer 1


Sentiment detection can be ambiguous & sometimes ill-defined. So only once you know your data is cleanly labeled, balanced & well pre-processed, I would then continue to re-modeling.

Detecting neutrals via thresholding off from a binary classifier is interesting but won't give you much lift because neutrals aren't necessarily the "absence of" or "cancelation of" positive/negative signals.

That said, there is still merit in enforcing the "spectrum nature" of this problem. You can learn a custom ensemble similar to OneVsOneClassifier but rather than majority voting, it could apply a rule to neutralize ambiguities & contradictions. This way most mis-classified positives or negatives will at least be less wrong. You can use any base estimator for this, including perceptron neural-nets.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.