I’m currently using a One vs Rest Random forest algorithm for multi class classification problem using Python, and I want to find the optimal threshold for each class, How can I do this with OVR (One-Vs-Rest) approach?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In general it's not possible to tune any threshold in multiclass classification:
- In binary classification, modifying the threshold means predicting more or less instances as positive, because the two classes are complement of each other (any instance which is not positive is negative and conversely).
- This doesn't work in the multiclass setting: for instance, an instance is predicted 55% class A. If the threshold is say 60%, what should be predicted instead? Class B predicted at 30%, or class C predicted at 15%?
The only way to reintroduce thresholds in a meaningful way would be to switch to multi-label classification, i.e. predicting every class as 'yes' or 'no' independently of the other classes (i.e. N independent binary problems). But this is a very different way to design the problem, since it means that an instance can have zero or multiple classes.
