I am working in the problem where the dependent variables are ordered classes, such as bad, good, very good.

How could I declare this problem in xgboost instead of normal classification or regression?



You can run 2 xgboost binary classifiers

  • 1 classifier classifies if sample is (good or very good)
  • 2 classifier classfies if sample is very good

  • if both true on unseen data classify as very good

  • if only 1st one true, second false classify as good both false=> classify as bad
| improve this answer | |
  • $\begingroup$ What to do if first false but second true? $\endgroup$ – Ben Reiniger Oct 29 '19 at 15:50
  • $\begingroup$ if both classfiers trained well, should happen only rarely and should be classified as bad. if need more tuning can output probabilities and compare probabilities instead of labels $\endgroup$ – alexprice Oct 30 '19 at 12:45
  • $\begingroup$ Indeed, this is probably a better situation than the regression setup in the other answer in the case of conflicting uncertainty. You could just output "I don't know," or if a decision is required, make sure the classifiers are probabilistic and well-calibrated. $\endgroup$ – Ben Reiniger Oct 30 '19 at 15:05
  • $\begingroup$ You can also use the prediction/probabilities of earlier labels as features for the higher labels. For example, the classifier 2 can be given the probability that classifier 1 already indicated it was at least 'good' as a feature $\endgroup$ – DrewH Feb 14 at 21:42

I think you can use a regression setup, e.g. bad=0, good=0.5, very good = 1 for labels, and then postprocess output of XGBoost, such as pred_value < 0.25 => prediction_label=bad, pred_value >= 0.25 and pred_value < 0.75 => prediction_label=good and so on.

| improve this answer | |
  • $\begingroup$ +1, but the two-classifier ordinal approach seems more flexible. $\endgroup$ – Ben Reiniger Oct 30 '19 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.