I have a C5.0 model that is trained to predict binary class (C1/C2) on a dataset with 20 features. The model is configured to perform boosting (10 trials) and it has a
miss-classification cost function (
100:1 where 100 is the cost for miss-classifying a Negative sample as Positive and 1 is the cost for miss-classifying Negative as Positive).
Looking at predicted probabilities generated by the model, I can see that it ranges from 0 to 1 for each class. i.e, I have instances where the predicted class (C1) has a probability lower than 0.5 ( for example: predicted class=C1 and predicted probability=0.1 ) - This is where the question arises: if
P(C1) < 50% why is it classified as C1 (there are only two classes, C1 and C2)
Based on my understanding of decision trees, the predicted probabilities are often generated based on the percentage of test cases on the leaf node that were labeled in each class divided by the total number of instances hitting that leaf node. This method will dictate the probability for two classes must sum up to 1.
My question is why does the model classify an instance in class C1 if it has only got 0.1 confidence in it. Does a predicted probability of 0.1 on class C1 mean that there is a 0.9 confidence in it belonging to class C2? If so, why does it classify an instance in class C1 is it has less than 0.5 confidence in it?
My own theory is that this might be due to boosting and miss-classification cost and the way they are influencing the predicted class.