I am trying to train a few Machine Learning (ML) algorithms such as SVM, NB and Random Forest to do binary classification on disaster tweets. During this project, I want to train ML algorithms for a combined disaster dataset. However, I want to apply it to individual disasters separately (e.g, train on multiple disasters, apply to earthquake; train on multiple disasters, apply to flood). I have the combined dataset in one
DataFrame and individual disasters in separate
In this case, how should I split my train and test datasets? At the moment, I split my combined dataset and apply ML algorithms.
from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(all_disaster['cleaned'], all_disaster['target'], test_size=0.33, random_state=42)
But, I feel that is not the proper way, as I want to apply it into individual disasters.