I'm doing sentiment analysis of tweets related to recent acquisition of Twitter by Elon Musk. I have a corpus of 10 000 tweets and I'd like to use machine learning methods using models like SVM and Linear Regression. My question is, when I want to train the models, do I have to manually tag big portion of those 10 000 collected tweets with either positive or negative class to train the model correctly or can I use some other dataset of tweets not relating to this topic that's already tagged to train the model for sentiment analysis? Thank you for your answers!
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
If you train a model, you train it to make it work in a more general situation (e.g. when you use the test set, unseen data, to evaluate your model, you just compute what is called generalization error).
You don't train a model to work only with data you trained it with, but to work good with unseen data (else it means you overfitted, so the model is useless).
So you can train a model for sentiment analysis on some tweets dataset (you can find a lot of them online, with all labels, so you can compute metrics with this data to make sure it works), and then use this model to make predictions on your own data. It will obviously act as an unsupervised task (I mean over your 10k tweets), since you haven't labels (so you can't compute metrics), but if the model was trained in the right way, it will works.
$\begingroup$
$\endgroup$
Depending on the language of the tweets you collected, and the availability of pre-trained sentiment analysis models for this language.
You should aim for models trained on the most similar domains, typically models that were trained with social media text, as it is quite different than other domains (e.g. news, or articles). However, you will always have a problem with validation, since you do not have ground truth data.
To get around this problem, you could encode the tweets using a pre-trained encoder (e.g. universal sentence encoder or BERTweet). Then cluster the encoded tweets. Optionally, you could project the tweets first using UMAP (faster) or t-SNE (less efficient with large datasets). You could then label few tweets from each resulting cluster and propagate the labels for most similar tweets. This approach would work since the encoders used are trained for semantic similarity tasks, so they are encoding tweets similar in meaning into similar vectors, and then the UMAP projections further brings similar vectors closer (due to the reduced dimensionality). You could then use the propagated labels as predictions, and the few labeled tweets from each cluster as ground truth.
This approach has been validated and used multiple times in NLP and computational social science literature. An example here.
