I have a dataset containing headlines and sentiment related to those headlines. The headlines have been filtered out from another bigger dataset using the following criteria: keep the ones that have a very negative or a very positive sentiment. At the end, I have a dataset with a very positve and a very negative sentiment headlines.
My goal is to create a deep learning classifier using tf and keras in order to classify new observations into three class: positive sentiment, negative sentiment and neutral sentiment. In other words, my goal is to use a binary labelled dataset to create a classifier that outputs a 3-label classification and I want to do it by predicting the probability of a headline to be positive or negative.
If the predicted probabilities of a headline are:
p(positive) = 80% p(negative) = 20%
than the headline is positive. But if:
p(positive) = 50% p(negative) = 50%
than the headline is neutral.
What do you think?