I have big table in dataframe (600k rows) which has y column (the variable I want to predict) and other 4 other columns that are the X. I have run RF regressor and I got score of 0.87 when I run it on the train and test.
However, when I tried to predict another set of data (which is very similar, with 1M rows) I got score of 0.65. So I assumed that is overfitting. when I tried to understand why it hapenns, I went back to the distribution of the y column, which looks like this:
my question is, can it be that because that my data does not have normal distribution (or very skewd...) my model preformance is bad? Do all variables need to have normal distribution? how does the score of the random forest regrssion is calculated? id value is 0.25 and predict is 0.26 does it count as correct prediction?