0
$\begingroup$

I have been working on a multilabel classification problem. I am using Python machine learning libraries to implement the classification algorithms. For the cross-validation, I am using repeated K-Fold cross-validation. I have experimented with SVM, Logistic Regression, Random Forest, Decision tree, K-Neighbour, and Naive Bayes and using Binary Relevance, Classifier Chain, and Label Powerset transformation methods for all of them.

I noticed, for Classifier Chain, SVM, Logistic Regression, Random Forest, and K-Neighbors are always achieving same subset accuracies and hamming losses. For Label Powerset, SVM, Logistic Regression, and Random Fores are achieving the same scores. However, for the binary relevance, all scores are different. No matter, what random seed I use or how many repetitions of cross-validations I run, they always end up scoring the same. I was wondering if this is a normal occurrence or they are suffering from class imbalance or overfitting or anything of that sort problems?

P.S. I am a beginner. Sorry for bothering with a naive question.

Improve this question
$\endgroup$
2
  • $\begingroup$ The question is very general. It seems like you are trying to ask what are the causes of lower accuracy? There are several parameters that affect the performance and accuracy of any algorithms including multilabel classification - type of data used, how have you normalized your dataset, type of tuning parameters used. In order to give specific answer, I suggest you add more context to your question. $\endgroup$
    – Suresh KUMAR Mukhiya
    Commented Dec 3, 2018 at 8:42
  • $\begingroup$ No, actually scores are good. Hamming loss is around 10% and subset accuracy is around 50%. I am doing multilabel classification of texts. I did clean up text, stemmed it. One thing to note, there is a class with 51% example instances assigned to it. $\endgroup$
    – Sayem Imtiaz
    Commented Dec 3, 2018 at 9:32

1 Answer 1

Reset to default
1
$\begingroup$

Same accuracies and losses for multilabel classification usually refer to the fact that model just chooses majority class. Please check the confusion matrix. Definitely, it is not a normal occurrence and you need to deal with it.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your feedback. I will look in this direction. But, a thing to note in my results is, scores are good. Hamming loss is around 10% and subset accuracy is around 50%. I am doing multilabel classification of texts. I did clean up text, stemmed it. Although there is a class with 51% example instances assigned to it. $\endgroup$
    – Sayem Imtiaz
    Commented Dec 3, 2018 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.