I am trying to predict the glucose values of the patients for example values like 45,256,115 etc. based on some features. Currently I am calculating the accuracy in means of RMSE,MSE,R². Is there any way to calculate the accuracy in means of a confusion matrix by setting a threshold value like ±10 for the value predicted. For example if the actual value is 110 and my prediction is 100 then we can tell that's it's kind of accurate since 100+10=110 where 10 is the threshold value. Any suggestions and some code to elaborate it will be very helpful.
$\begingroup$
$\endgroup$
5
-
1$\begingroup$ What is it that you would like to do? Is it that you are not looking for a continuous outcome but rather something like "glucose is over a threshold", or is it that you would like a more in-depth view of the individual predictions and actuals? $\endgroup$– S van BalenCommented Feb 19, 2020 at 8:48
-
$\begingroup$ Yeah first I focused on continuous outcome but now I want to test how accurate the value can be predicted along with the threshold value without using regression scoring metrics. $\endgroup$– HelloWorldCommented Feb 19, 2020 at 12:32
-
$\begingroup$ How about reframing it in a classification problem? $\endgroup$– S van BalenCommented Feb 19, 2020 at 15:04
-
$\begingroup$ I don't think it can be reframed since I need to predict the blood glucose value.. any suggestion to reframe it ? $\endgroup$– HelloWorldCommented Feb 20, 2020 at 2:20
-
$\begingroup$ How about? glucose_value > THRESHOLD ? 1 : 0 Or binning it if you need more more resolution $\endgroup$– S van BalenCommented Feb 20, 2020 at 8:11
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
I believe that there's no actual function in sklearn library that does that. However, that concept is relatively easy to implement: you can loop and compare each y_prediction with the respective y_true_value and increment TN, TP, FN, and FP accordingly to a certain threshold. For example:
if abs(y_pred[i] - y_true[i]) <= threshold:
TP += 1
elif abs(y_pred[i] - y_true[i]) > threshold:
TN += 1
...
Don't forget to reverse transform the scaling on y_true and y_pred values first.
-
$\begingroup$ Yeah thanks that helps. By the way do you know a way of calculating the opimum threshold value? $\endgroup$ Commented Feb 20, 2020 at 2:18
-
$\begingroup$ you're welcome! I think it depends on the nature of the problem, and tbh you might be in better position than me to know what threshold makes sense or could be optimal. $\endgroup$ Commented Feb 20, 2020 at 7:39
-
$\begingroup$ Yeah that makes sense but is there any formula to calculate the threshold value? $\endgroup$ Commented Feb 20, 2020 at 8:01