I have a dataset (600 rows) composed of two columns:
-Summary: which contains the text of a document
-Keywords: which contains the keywords that identify that document.
Summary KeyWords_in_Array_wo_insurance 0 court sanction scheme transfer insur reinsur b... [insolvency] 1 immedi custodi sentenc month week impos direct... [administration of justice, civil evidence, se... 2 motorist injur hit run collis car identifi dri... [negligence, road traffic] 3 claimant given permiss continu claim compani a... [insolvency, civil procedure] 4 court gave guidanc approach taken applic relea... [civil procedure, costs] 5 plaintiff solicitor entitl declar life critic ... [trusts] 6 claimant insur establish requir standard road ... [personal injury, torts] 7 minimum indemn requir institut charter account... [arbitration, civil procedure, costs, accounta... 8 applic secur cost court could take account eve... [civil procedure, insolvency, cpr, costs]
I want to classify a summary in specific keywords. The keywords are not mutually exclusive.
My code is:
X_train, X_test, y_train, y_test = train_test_split(df_final["Summary"], df_final["KeyWords_in_Array_wo_insurance"], test_size=0.20, random_state=42) mlb = MultiLabelBinarizer() y_train_mlb = mlb.fit_transform(y_train) classifier = Pipeline([ ('vectorizer', CountVectorizer()), ('tfidf', TfidfTransformer()), ('clf', OneVsRestClassifier(LinearSVC()))]) classifier.fit(X_train, y_train_mlb) y_predicted = classifier.predict(X_test) all_labels = mlb.inverse_transform(y_predicted) y_test_mlb = mlb.transform(y_test) print("Accuracy = ", accuracy_score(y_test_mlb,y_predicted))
I am getting a low accuracy score: 20%
Therefore I am thinking that my classification is not good enough.
The reason it might be that some keywords are used only once. For example, the keywords "animal" or "partnership" or "succession" are used only in 1 row. (Meaning, they are assigned to only one summary)
I think, therefore, that when I split the dataset in training and test, some "lonely keywords" enter the test dataset but not the training one. Thus, the model will never be trained on them.
Is this the reason why my accuracy is so low?
Or am I doing something else wrong?