I have been working on a predictive model. With each prediction, we need to provide a score to express the confidence about our prediction. So I am looking at prediction interval (PI). In linear regression, I believe these can be obtained and well-documented. However, I am yet to find much reference for non-linear regression (such svr, gbr or other blackbox method for regression). Two methods that I have seen are given below:
1) Using bagging, we can generate many point prediction of each new data point, and then we get the interval from the distribution of these predictions around each new point.
2) using Quantile regression to get the upper and lower bound of the new point.
Personally, I do like the bagging method, although I don't feel very convinced. Hence, I am reaching out to the community to get a general opinion or some other ideas that I haven't seen so far.