I am working in the problem where the dependent variables are ordered classes, such as bad, good, very good.

How could I declare this problem in xgboost instead of normal classification or regression?



2 Answers 2


You can run 2 xgboost binary classifiers

  • 1 classifier classifies if sample is (good or very good)
  • 2 classifier classfies if sample is very good

  • if both true on unseen data classify as very good

  • if only 1st one true, second false classify as good both false=> classify as bad
  • $\begingroup$ What to do if first false but second true? $\endgroup$
    – Ben Reiniger
    Oct 29, 2019 at 15:50
  • $\begingroup$ if both classfiers trained well, should happen only rarely and should be classified as bad. if need more tuning can output probabilities and compare probabilities instead of labels $\endgroup$
    – alexprice
    Oct 30, 2019 at 12:45
  • $\begingroup$ Indeed, this is probably a better situation than the regression setup in the other answer in the case of conflicting uncertainty. You could just output "I don't know," or if a decision is required, make sure the classifiers are probabilistic and well-calibrated. $\endgroup$
    – Ben Reiniger
    Oct 30, 2019 at 15:05
  • $\begingroup$ You can also use the prediction/probabilities of earlier labels as features for the higher labels. For example, the classifier 2 can be given the probability that classifier 1 already indicated it was at least 'good' as a feature $\endgroup$
    – DrewH
    Feb 14, 2020 at 21:42

I think you can use a regression setup, e.g. bad=0, good=0.5, very good = 1 for labels, and then postprocess output of XGBoost, such as pred_value < 0.25 => prediction_label=bad, pred_value >= 0.25 and pred_value < 0.75 => prediction_label=good and so on.

  • $\begingroup$ +1, but the two-classifier ordinal approach seems more flexible. $\endgroup$
    – Ben Reiniger
    Oct 30, 2019 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.