I'm currently using Google's BERT pre-trained sentiment analysis model that is trained on an IMDb pos/neg review dataset. I'm using this model to predict whether tweets are positive (bullish) or negative (bearish). While the model is accurate when plugging in my own test data (F1 Score ~86%), the classification itself is not accurate. Tweets that are undoubtedly positive/bullish, and not classified as so. Perhaps this is because the language in the investment world is different than a movie review - which uses universally recognized positive/negative words and/or sentences.
The same is true when I take my tweet dataset and use Vader SentimentIntensityAnalyser to parse pos/neg tweets into separate folders.
So my question is... since the language that is used for telling whether a stock is bullish/bearish is uniquely different from that of an Amazon review, or movie review, would it be optimal for me to manually classify my dataset into positive (bullish) and negative (bearish) datasets?