I am working on a multiclass classification which is to assign medical related queries of web search to certain departments of hospital.My classifier is based on the fastText.

I found for most conditions, the result is good enough say recall is 0.8 for Nephrology. However, for just one department, Dermatology, the recall is pretty low,like 0.5. Unfortunately, this label has most samples in the test data.

How can I improve the recall of one class while maintain the performance of other classes? Will ensembling method work?

  • $\begingroup$ Hello, welcome to the site! I would like to know what technique you have used for separating the data into test and train? $\endgroup$ – Toros91 Jan 15 '19 at 5:56
  • $\begingroup$ @Toros91 I just shuffle the original data and down-sampling the labels with too many samples and seperate train-test by 8:2 for each label. Acturally, I tried to get an balanced data by setting up a threshold for ~30000, any label with more than 30000 samples will be down sampled. However, in the original data, there are about 240000 samples with label Dermatology. I am not confident of my method to choose data, do you think it causes the problem? $\endgroup$ – leakey Jan 15 '19 at 6:12
  • 1
    $\begingroup$ so now when you do that I would suggest you to Stratified sampling which would help you in doing the sampling better and you wouldn't have problem while classification. Did you check if the levels in the test and train are same? $\endgroup$ – Toros91 Jan 15 '19 at 6:16
  • $\begingroup$ @Toros91 Thanks! I used disproportionate stratified sampling, maybe I should try to use proportionate stratified sampling for large classes. Appreciate your help. $\endgroup$ – leakey Jan 15 '19 at 6:26
  • $\begingroup$ One more suggestion is if you find a level less significant then it is not necessary for you to keep it in the training or testing set. You can use a new category called other when it is not classified into any class. can be segregated into that $\endgroup$ – Toros91 Jan 15 '19 at 6:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.