0
$\begingroup$

I am wondering what is the best way to calculate the uncertainty for my categorical predictions. I have created a model that predicts what rating a movie is getting based on certain keywords and the amount of times these key words are mentioned. I have trained the model with a small data matrix and now I am running it though a bigger data matrix that has data from over 20,000 reviews. I have used different models and different variations of my dataset (feature engineering and feature selection) and now that I have my results from my best model/dataset combination, I want to learn how to calculate the uncertainty of my results per rating. Could I use the confusion matrix or other metrics (sensitivity etc) to calculate the uncertainty?

Example Predictions: Rating1: 1000 Rating2: 25 Rating3: 59 Rating4: 599 Rating5: 3569

Thank you!

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ What do yo mean by "uncertainty" ? Is it the average error of your predictor ? Or the variance of your predictor error (see bias-variance tradeof) ? Or the self-estimated confidence that your model has about its predictions ? $\endgroup$
    – xtof54
    Dec 22, 2020 at 14:49
  • $\begingroup$ I am looking more towards the self-estimated confidence that the model has for the predictions, thanks for the follow-up $\endgroup$
    – brad987
    Dec 22, 2020 at 20:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.