1
$\begingroup$

I have a small dataset of 30 rows and 5 columns (4 features and 1 class). The classifier is used to give the likelihood of occurrence of an incident. thus, the class variable gives the probability of occurrence.

How can I measure the uncertainty and prediction errors of my classifier?

I have read a post here about measuring the uncertainty of prediction. However, it is tacking the issue of multiclass classification!

$\endgroup$
1
$\begingroup$

You can check the probability returned by your model. Just be aware that probability is not uncertainty but if you calibrate your probabilities, you can have an idea of the belief of event realisation from a frequentist point of view.

For the error of prediction, with a calibrated classifier if you have a probability of 0.8 to belong to a certain class, you can expect that there is a 80% chance that your sample actually belongs to this class.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.