Say I have an imbalanced data set, and I decided to over/undersample it during model training. I would then like to predict on new records but using the original, true imbalance in the classes as an apriori for the model.

i.e my classes are distributed in a ratio of 1:100, I undersample to 1:2, but would like the model to know that class A is very rare so be careful in predicting it.

For logistic regression I am familiar with a method of intercept correction, detailed here:

My question is: what if I use deep leaning instead of logistic regression? My reasoning is that since the last layer of the NN is basically the same a logistic regression, I can use the same method for the intercept of this layer. Can you think of any objections? any reason this doesn't make sense? If so, how would you go about correcting the model?

Also, is there a generalized method for intercept correction for a case when I have more than 2 classes?

  • $\begingroup$ Welcome to this site! I agree with you, since the proposed correction does not make any assumption about the feature space $\boldsymbol{x}$, therefore, all layers from input $\boldsymbol{x}$ to the last layer, act as a transformation that produces a new feature space $\boldsymbol{x'}$, and the rest is the same. $\endgroup$
    – Esmailian
    Apr 1 '19 at 17:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.