0
$\begingroup$

I have a classification model (BERT) that classifies sentences as either question or normal sentences. But whenever a sentence has "how" word, the model chooses "question" class. How can I solve this issue? (I have a very big dataset.)

$\endgroup$

1 Answer 1

2
$\begingroup$

Very likely, the majority of the sentences which contain "how" in your training data are labelled as question. It's probably a problem of representativity of the training set, because otherwise the problem wouldn't be this specific. But note that it's likely that your training data contains other issues a well, possibly there are errors in the labels.

$\endgroup$
5
  • $\begingroup$ Thank you. But isn't this a very common problem? When I researched about it I couldn't see anything. I mean it does seem like I can face this issue any time. Why is there isn't any thread about this do you think? $\endgroup$
    – canP
    Commented Sep 29, 2022 at 20:52
  • 1
    $\begingroup$ Sure, it's very common: it's about the quality of the dataset used to train the model, in this case whether it's sufficiently representative. Often people simplify this problem by @canP saying that it's only a matter of size, assuming that if it's large enough it's probably representative. But this issue is something to take care of at the creation/collection of the dataset, this part doesn't involve any ML and is considered boring... probably the reason why it's not discussed much in ML books, I guess. $\endgroup$
    – Erwan
    Commented Sep 29, 2022 at 21:54
  • $\begingroup$ Yea... It's like: For example, there are 2 classes: Cat and dog. There is always a watermark in the corner of the picture in dog-tagged pictures. So imagine this: there is a cat photo with a watermark on it. I have no idea what should i do if my model classifies the model as a dog. Maybe I will add watermarks to cat-labeled train data too. But I don't know if it's a good solution. $\endgroup$
    – canP
    Commented Sep 30, 2022 at 14:58
  • $\begingroup$ @canP the principle in supervised learning is that the model is going to be applied to some data with the same distribution as the training data. This means that if there is always a watermark on every dog image during training, it is assumed that the same condition applies during testing (or production). If it doesn't, then it means that the training data was not representative enough: it should at least contain some dog images without watermark, in order for the model to learn that watermark doesn't imply dog. $\endgroup$
    – Erwan
    Commented Sep 30, 2022 at 19:37
  • $\begingroup$ Yes, its very logical. So i should develop my understanding of "representative data" more i guess. Thank you! $\endgroup$
    – canP
    Commented Sep 30, 2022 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.