Xgboost multiclass monotonic constraints

Question

I have a problem where i have a variable price and i need to classify this price as winning/non-winning. If price grows, probability should monotonically go down. I use a monotonic constraint that works fine (the xgboost library parameter https://xgboost.readthedocs.io/en/stable/tutorials/monotonic.html)

I'm using xgboost python package version 2.0.1.

Now, i'm trying to upgrade my solution to a multiclass classification problem. Now, for each price, I want to know the probability of belonging to class 1, winning, class 2, be second and class 3, rest of positions.

Using xgboost 'objective' = 'multi:softprob' i get fairly good results. However, the 'monotone_constraints' = {"price": -1} has no effect.

Is there to ensure the monotonic constraint in the multiclass classification problem? Is xgboost the best library for this issue? is it even needed to ensure the monotonic constraint?

Answer 1

The monotonic constraint doesn't make sense in a multiclass classification problem.

You want the relationship to be: as price increases, the probability of being class 1 decreases (and the probability of class 2 and 3 should increase). However, consider that multi-class classification will treat a prediction of class 3 (for a true class 1) just as bad as a prediction of class 2, even though class 2 is closer to the true rank than the third class.

What you have is an ordinal regression problem, where the labels have an associated ranking between them. Some models can handle this kind of problem, but as far as I can tell, there are no readily available gradient boosting algorithms for this. It would appear that most just treat this as a standard regression problem where the target is the true ranking of the observation (i.e. 1, 2, 3, 4, ...) for 1st place, 2nd place, third place, etc. when using gradient boosting algorithms. In that approach, the monotonic constraint will work as intended (as will it if you were to use an ordinal regression model).