I have an annotated text dataset on hate speech. Simply put, the dataset consists of a column called
text which includes a piece of text, and a column called
label which can be either
0 (non-hateful) or
1 (hateful). Let's call this dataset
I am using this dataset to train a very simple classifier using Logistic Regression. Code:
dataset_1_df = pd.read_csv(d_1_path, sep='\t') text = dataset_1_df.text X = text y = dataset_1_df['label'].astype(int) X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, test_size=0.1) clf = Pipeline([ ('vect', CountVectorizer(max_features=10000, ngram_range=(1, 2))), ('tfidf', TfidfTransformer(norm='l2')), ('clf', LogisticRegression()), ]) clf = clf.fit(X_train, y_train) y_preds = clf.predict(X_test)
This classifier works fine. I can produce a classification report like this:
report = classification_report(y_test, y_preds) print(report)
precision recall f1-score support 0 0.87 0.70 0.78 410 1 0.94 0.98 0.96 2069 accuracy 0.93 2479 macro avg 0.91 0.84 0.87 2479 weighted avg 0.93 0.93 0.93 2479
And I can now make predictions like this:
print(clf.predict(['I hate you'])) print(clf.predict_proba(['I hate you']))
 [[0.0749381 0.9250619]]
I have another dataset, let's call this
Dataset B, independent to
Dataset B is also an annotated text dataset on hate speech with the same columns.
What I want to do is to test the model I trained using
Dataset A on
Dataset B. I.e., I want to produce a classification report on my model's performance on
Dataset B. Can I do that? If yes, how?
Clarification: I do not want to retrain my model using
Dataset B. I want to see how good its performance is on the new dataset.
There is this question that looks similar based on the title but does not ask what I'm asking.
There's is also this question which asks how to make predictions. However, I know how to make predictions, what I want to do is to test my trained model on a totally new dataset.