I am working on building a sentiment analyzer, the data I would like to analyze is social media data from twitter, once I have created a the model I want to integrate it into a simply webpage.
I have tried two options:
Create my own model from scratch, meaning train a word2vec model to perform word embedding, convert my labelled dataset into vectors and train them using Logistic regression, Random forest or SVM.
Fine tune a BERT model using my dataset.
option 1.. Using word2vec and SVM I was able to get the following results:
precision recall f1-score support
0 0.74 0.67 0.70 1310
1 0.77 0.82 0.79 1716
accuracy 0.76 3026
macro avg 0.75 0.75 0.75 3026
weighted avg 0.75 0.76 0.75 3026
option 2.. I fined tuned BERT using the following code link and was able to achieve the following results after 100 epochs:
precision recall f1-score support
0 0.68 0.65 0.66 983
1 0.74 0.77 0.75 1287
accuracy 0.71 2270
macro avg 0.71 0.71 0.71 2270
weighted avg 0.71 0.71 0.71 2270
I used the same dataset for both option 1 and 2, BERT used a smaller subset for validation
What I would like to know:
Is there any advantages in going with option 1.? Does BERT have any disadvantages when it comes to data from social media (data is rather unclean and a lot of slang).