I have built a regression model to estimate prices given some attributes. Prices are expressed in million USD, with average of 4.4 million USD. I need to provide an estimate of the RMSE and MAPE of the model. The average RMSE and MAPE of my model seem too high, with values of 3.4 million USD and 515 % respectively. However, looking at the RMSE and MAPE distribution in relation to the prices, I can see that there are a few, very low prices, that are difficult to predict and cause the very high mean error estimate. The scatterplot below shows price on the x axis and MAPE on the y axis
If I eliminate those 9 'outliers', the RMSE and MAPE go down to more reasonable values, namely 0.95 million USD and 26.7% respectively. My questions are:
- should I eliminate those 'outliers' (i.e. very low prices) before training?
- or should I train the model to predict all the data, and then exclude the outliers from the final RMSE/MAPE estimation?
- or is there any other best practice in providing the most realistic estimation of the error
- or, is there any other error metric that is robust to outliers?
Thank you very much