Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have a small dataset of 30 rows and 5 columns (4 features and 1 class). The classifier is used to give the likelihood of occurrence of an incident. thus, the class variable gives the probability of occurrence.
How can I measure the uncertainty and prediction errors of my classifier?
I have read a post here about measuring the uncertainty of prediction. However, it is tacking the issue of multiclass classification!
You can check the probability returned by your model. Just be aware that probability is not uncertainty but if you calibrate your probabilities, you can have an idea of the belief of event realisation from a frequentist point of view.
For the error of prediction, with a calibrated classifier if you have a probability of 0.8 to belong to a certain class, you can expect that there is a 80% chance that your sample actually belongs to this class.
