I have been looking for some methods to improve my prediction model, but could not find any so far. I have data, including 10 numerical features, that I use for prediction. I used a random forest regression model and it works perfectly on the train set (some part of perfection comes from over fitting).
However, the model is not doing a good job for data points that some of their features are not within the range of the sample data. For example, parameter x is between 0 and 2000 in the sample but the new data point that we want to predict has a value of 3000 for x.
I know this is due to data limitation, but I wonder if there is any way that I can generate samples or improve the prediction?
I wanted to build a simple model excluding this parameter, but the problem is that this is the most significant parameter in my model.
Any hints are appreciated.