XGBoost multiclassification interpreting predicted probabilities

Question

Let's consider an example. I have patient level data, their symptoms, reading from various medical tests. Based on that, I have built a binary classifier given patient data to classify if they are likely to have a disease, and if so, I'd want to manually run medical tests on them. For that case, we look at the disease class and rank order the predicted probabilities and select the riskiest patients to have the disease.

A few months later, say the disease evolves into a different strain. So we already have an imbalanced class of healthy vs infected patients. Not the new strain will be still a smaller population. Apart from that, the symptoms, various test readings will also be different. So translating this to a multi-classification problem makes sense. When you build a multi-classifier, and you have three probabilities, how would you go about rank ordering the same set of patients?

If you're interested in the risk of patients likely to be infected, taking the max of the either of the strain of disease a good way to proceed or is there a method to choose them based on their individual probabilities?

Edit: To bring a little more clarity. The multi:softprob gives the probability for each of the classes. If you have three classes, it will give three probabilities for each class summing up to 1. More details are here.

Given that we will get multiple probabilities for each row/patient from the example above, how do you go about choosing the final probability to rank order risky patients.

Answer 1

Multiclassification with xgboost is normally done One vs All. Meaning that first you try to predict if it belongs to class one or not. Then two second class or not...

You can understand as the probability of belongin to one class and not to the rest.

Let's put an example for better understanding: say that you have a ton of patients that have 3 illnes: Corona Virus, Sars and HIV.

You build your multiclassification model with xgboost or whatever and for a new patient you do a prediction and check the probabilities:

Corona [0.8] Sars [0.1] HIV [0.01] Not sick [0.09]

What you do from here is select the most probable cause (corona) and that is your final class.

If you do multilabel you can get something like this too Corona [0.9] Sars [0.20] HIV [0.999]

Meaning that even if it has a high probability of getting corona it also has a higher probability of having HIV. In multiclassification is different from multilabel where you can have a patient that has both of the sickness

From the sklearn documentation:

The one-vs-the-rest meta-classifier also implements a predict_proba method, so long as such a method is implemented by the base classifier. This method returns probabilities of class membership in both the single label and multilabel case. Note that in the multilabel case, probabilities are the marginal probability that a given sample falls in the given class. As such, in the multilabel case the sum of these probabilities over all possible labels for a given sample will not sum to unity, as they do in the single label case.

In summary, if sickness are independent it will predict the probability of the patient having one sickness, there has to be a class not sick just if there is the possibility for the patient not being sick

Note: In NN is different