In the kaggle forums I found an example model where someone was using XGBRegressor for a binary (0/1) classification problem (sorry, cannot find the link any more). This was for a competition where output is measured by AUC, so only ranks mattered.

I then tried this for a different problem with my data and also found a slight improvement using the Regressor instead of the Classifier.

Some of my predictions where negative, but it would be no problem to shift them to the [0,1] interval.

So is using regression for binary classification a common approach, and what are the pros and cons?

  • $\begingroup$ In regression I can do confidence interval, coefficient estimates, p-value, likelihood ratio test etc. But then I would have to estimate a threshold for classification. $\endgroup$
    – SmallChess
    Feb 3, 2017 at 9:34
  • $\begingroup$ Short answer: yes, it's awfully common. Logistic regression for binary classification is everywhere. $\endgroup$
    – SmallChess
    Feb 3, 2017 at 9:34
  • $\begingroup$ @StudentT not sure if I was clear, but I mean LINEAR regression for classification. $\endgroup$
    – spore234
    Feb 3, 2017 at 9:38
  • $\begingroup$ Ok. I never used linear regression for binary classification. Someone else might give you better response. $\endgroup$
    – SmallChess
    Feb 3, 2017 at 9:39


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.