Overview: Dataset is small and a bit messy and the task is to classify 5 classes wherein the targets are ordinal.

Feature Engineering and Selection, Model Tuning, etc. did not produce acceptable models.

Question: I have arrived at this point where I want to introduce sampling bias to reinforce our hypothesis regarding the target classes. When is this acceptable, or is this even acceptable given that you will be transparent about it?

What I did was to carefully select datapoints that align with our hypothesis on the lower minority classes (0 and 1) and use all the data for (2,3 and 4). This greatly improved the classification metrics and logloss.

  • 1
    $\begingroup$ whenever you want to commit a fraud. $\endgroup$
    – lpounng
    Apr 16 at 4:51
  • 2
    $\begingroup$ At that point if you just want to reinforce our hypothesis regarding the target classes why even bother with data? Just state your hypothesis and then prove it using authority/common sense/hearsay. $\endgroup$ Apr 16 at 5:47
  • $\begingroup$ I deleted some comments; keep things constructive and civil, please. $\endgroup$
    – Ben Reiniger
    Apr 17 at 12:49
  • $\begingroup$ I think the comments represent the obvious answer here. @easymoneysniper is there some additional context that makes you think otherwise? What's the use case of this model? $\endgroup$
    – Ben Reiniger
    Apr 17 at 12:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.