I am a newbie when it comes to machine learning. I am trying to get hands on experience by analyzing different supervised learning algorithms using scikit-learn library of python. I am using the sentiment140 dataset of 1.6 million tweets for sentiment analysis using various of these algorithms.
I don't know if it is a stupid question, but I was wondering whether if it'd be possible to classify into three classes (positive, negative and neutral) when you've only trained over two classes (positive and negative). The sentiment140 training set consists of only two classes (positive and negative) of 1.6M tweets in total but their test set consists of 500 tweets over three classes (positive, negative and neutral), so it got me thinking.
So is this possible? If yes, how do I proceed to identify the neutral tweets? Intuitively, I can use to find the conditional probability of each classified tweet in the test set over each class (using predict_proba) and tell if it's neutral if it is below a certain threshold (say less than 0.7) for both the positive and negative classes. Is this the right way to go?