I have data with 4 classes: $c_1, c_2, c_3, c_4$.
I'd like to create a classifier which has different scaling for the loss function per class combination:

$$ \begin{bmatrix} 0 & l \left( \hat{c}_{1}, {c}_{2} \right) & l \left( \hat{c}_{1}, {c}_{3} \right) & l \left( \hat{c}_{1}, {c}_{4} \right) \\ l \left( \hat{c}_{2}, {c}_{1} \right) & 0 & l \left( \hat{c}_{2}, {c}_{3} \right) & l \left( \hat{c}_{2}, {c}_{4} \right) \\ l \left( \hat{c}_{3}, {c}_{1} \right) & l \left( \hat{c}_{3}, {c}_{2} \right) & 0 & l \left( \hat{c}_{3}, {c}_{4} \right) \\ l \left( \hat{c}_{3}, {c}_{1} \right) & l \left( \hat{c}_{3}, {c}_{2} \right) & l \left( \hat{c}_{3}, {c}_{3} \right) & 0 \end{bmatrix} $$

Where $l \left( \hat{c}_{i}, {c}_{j} \right)$ is the loss factor for estimating the $i$ th class where the sample has the $j$ th class.

The motivation is to have a larger loss for some mistakes.

Some of scikit-learn models (RandomForestClassifier, HistGradientBoostingClassifier) allows setting a value per row by class_weight.
It is also supported by lightgbm's LGBMClassifier.

Is there way to achieve something arbitrary as the matrix above for xgboost / scikit-learn or lightgbm?
Is there any other classifier in the Python eco system to support such a generic matrix? It seems to be supported to some degree in the deep learning case: Loss for ordered multi class data in classification.

Is seems to be very useful for many cases in classification.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.