0
$\begingroup$

I have trained a regression model and obtained a pandas series of the predicted values. I am working on a "calculator" that will be able to return a predicted value after entering an input sample. I would like to inform an end user, how this predicted value can differ (something like range: min | predicted value | max).

The input dataset on which the model was trained has many outliers, hence I suppose that adding / subtracting a constant value like mean_absolute_value to / from the predicted value is not the best idea.

What is the best approach to this problem? Which metric should I use? Ideally from some sklearn module.

$\endgroup$

1 Answer 1

0
$\begingroup$

I think you need to compute/output confidence intervals for each prediction because, often, you are not given the true data at inference. Basically you decide how much wide your interval should be, the wider the less confident. The intuition is that the true value is contained within the interval with some probability $p$, whereas your prediction is always inside the inteval.

For practical implementation, assuming linear regression, you may want to look at these two answers: 1 and 2.

Another way would be to directly use models that can estimate confidence, like Bayesian and Quantile regression, or even a Gaussian Process.

$\endgroup$
2
  • $\begingroup$ Thank you! I will read about that. One question arose: how this approach is better than the percent MAE? I've just found it on sklearn (link: scikit-learn.org/stable/modules/generated/…) $\endgroup$
    – Paulina
    Commented Jul 6, 2023 at 10:30
  • $\begingroup$ @Paulina If you have access to the true value, then you can use this metric but also MSE, RMSE, or even the $R_2$ coefficient. $\endgroup$ Commented Jul 7, 2023 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.