Text classification task, if data quantity is low but data quality is not low. We could use data augment methods for improvement.

But the situation is that data quantity is not low and data quality is low. (noise in the labels, or training data accuracy low)

The way I get the low quality data is by unsupervised methods or rule-based methods. In detail, I deal with a multi-label classification task. First I crawl web page such as wiki and use regex-based rule to mark the label. The model input is the wiki title and the model output is the rule-matched labels from wiki content.

  • $\begingroup$ What have you tried? Based on your tags, it seems that there is much more to your story than what you wrote in your post. $\endgroup$ – Valentin Calomme Jun 2 at 7:08
  • $\begingroup$ @ValentinCalomme Thank you. I will write more. $\endgroup$ – 不是phd的phd Jun 2 at 7:10
  • $\begingroup$ Can you specify what you mean by saying "data quality is low". Are you referring to missing values, or noise in the labels? $\endgroup$ – bonfab Jun 2 at 10:33
  • $\begingroup$ @bonfab noise in the labels $\endgroup$ – 不是phd的phd Jun 2 at 12:00
  • $\begingroup$ @bonfab Thank you $\endgroup$ – 不是phd的phd Jun 2 at 12:06

If the noise is not too large, a well regularized model should perform well.

Also ensemble methods could work well, since they reduce the variance of the model. Maybe also try an ensemble with an unsupervised method like clustering, to reduce the dependency on the labels.

Otherwise there have been methods developed that handle noisy labels https://stats.stackexchange.com/questions/218656/classification-with-noisy-labels.

| improve this answer | |
  • $\begingroup$ github.com/subeeshvasu/Awesome-Learning-with-Label-Noise $\endgroup$ – 不是phd的phd Jun 3 at 2:02
  • $\begingroup$ Thank you. What are the simplest methods for the label noise problem? $\endgroup$ – 不是phd的phd Jun 3 at 2:10
  • $\begingroup$ Do you think removing the wrong data predicted by trained model is a simple but effective method? $\endgroup$ – 不是phd的phd Jun 3 at 2:11
  • $\begingroup$ What the best method is, really depends on your data. If the noise is not too high, I would try an ensemble approach. Only remove data where you are very sure that the label is wrong. The more difficult, correctly labelled datapoints are the most important in a data set. Use explainability methods to analyze which patterns your model is picking up before you automize the cleaning of your dataset. $\endgroup$ – bonfab Jun 3 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.