1
$\begingroup$

I am trying to predict the glucose values of the patients for example values like 45,256,115 etc. based on some features. Currently I am calculating the accuracy in means of RMSE,MSE,R². Is there any way to calculate the accuracy in means of a confusion matrix by setting a threshold value like ±10 for the value predicted. For example if the actual value is 110 and my prediction is 100 then we can tell that's it's kind of accurate since 100+10=110 where 10 is the threshold value. Any suggestions and some code to elaborate it will be very helpful.

$\endgroup$
  • 1
    $\begingroup$ What is it that you would like to do? Is it that you are not looking for a continuous outcome but rather something like "glucose is over a threshold", or is it that you would like a more in-depth view of the individual predictions and actuals? $\endgroup$ – S van Balen Feb 19 at 8:48
  • $\begingroup$ Yeah first I focused on continuous outcome but now I want to test how accurate the value can be predicted along with the threshold value without using regression scoring metrics. $\endgroup$ – HelloWorld Feb 19 at 12:32
  • $\begingroup$ How about reframing it in a classification problem? $\endgroup$ – S van Balen Feb 19 at 15:04
  • $\begingroup$ I don't think it can be reframed since I need to predict the blood glucose value.. any suggestion to reframe it ? $\endgroup$ – HelloWorld Feb 20 at 2:20
  • $\begingroup$ How about? glucose_value > THRESHOLD ? 1 : 0 Or binning it if you need more more resolution $\endgroup$ – S van Balen Feb 20 at 8:11
2
$\begingroup$

I believe that there's no actual function in sklearn library that does that. However, that concept is relatively easy to implement: you can loop and compare each y_prediction with the respective y_true_value and increment TN, TP, FN, and FP accordingly to a certain threshold. For example:

if abs(y_pred[i] - y_true[i]) <= threshold: TP += 1 elif abs(y_pred[i] - y_true[i]) > threshold: TN += 1 ...

Don't forget to reverse transform the scaling on y_true and y_pred values first.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yeah thanks that helps. By the way do you know a way of calculating the opimum threshold value? $\endgroup$ – HelloWorld Feb 20 at 2:18
  • $\begingroup$ you're welcome! I think it depends on the nature of the problem, and tbh you might be in better position than me to know what threshold makes sense or could be optimal. $\endgroup$ – joelpires Feb 20 at 7:39
  • $\begingroup$ Yeah that makes sense but is there any formula to calculate the threshold value? $\endgroup$ – HelloWorld Feb 20 at 8:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.