I am recently engaged in a project of predicting the future blood glucose values of the patient. I was able to get a lower RMSE and MAE by using Gradient Boosting Algorithm. But although the error is low the predicted and actual values have a significant difference. For example, in the below image the blue line is actual while the orange is the predicted(using Gradient Boosting algorithm).
With the above prediction it's clear that it's not that accurate. So I wanted to predict the prediction interval with the upper bound of 0.9 and lower bound of 0.1, such that I will be able to tell that their next glucose values will lie within a specific range. I saw in an article that they have used gradient boosting algorithm to predict the intervals with the quantile loss function. The code I typed for predicting the prediction intervals is as follows,
LOWER_ALPHA = 0.1 UPPER_ALPHA = 0.9 lower_model = ensemble.GradientBoostingRegressor(loss="quantile", alpha=LOWER_ALPHA) upper_model = ensemble.GradientBoostingRegressor(loss="quantile", alpha=UPPER_ALPHA) lower_model.fit(X_train, y_train) upper_model.fit(X_train, y_train) predictions = pd.DataFrame(y_test) predictions['lower'] = lower_model.predict(X_test) predictions['upper'] = upper_model.predict(X_test) plt.figure(figsize = (20,20)) plt.plot(predictions['lower']) plt.plot(predicted) plt.plot(predictions['upper'])
What I got as a result is,
But I feel like I have done something wrong. Because the gap between the upper bound and lower bound is very high and for that type of prediction we don't need machine learning. For example if we look at the first point the upper bound lies somewhere around 215 and lower bound lies around 60, So as a result I what I can tell is that the next value is gonna lie within 60-215 range. What have I done wrong? And can anyone suggest a better way of calculating the prediction interval? Thanks in advance.