Generally model gets biased towards data_samples/target whose frequency is high in training data set. Is it possible during training that model gets biased towards low frequency training data set.

  • $\begingroup$ Could you please elaborate your question and problem? $\endgroup$ May 24 '19 at 4:46
  • $\begingroup$ We have a dataset of binary classifier. Where class 1 data is huge whereas class 0 is having very less data, i.e. data is skewed. During model training, its quite possible that model should be biased towards class 1 and its expected. I want to know is it also possible if model get biased toward class 0? $\endgroup$ May 24 '19 at 4:48

With structured data, you have in general 4 challenges:

(1) Missing data

(2) Outliers

(3) Cardinality

(4) Rare values (as a rule of thumb <5%)

Rare values in categorical variables tend to cause over-fitting, particularly in tree based methods. Ph.D. Data Scientist Soledad Galli has an amazing course on the subject (Udemy: "Feature Engineering". Below a screenshot from her course, but to be fair to her, I'm not going to post the solution.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.