Appropriate model metric for a truncated response variable?

Question

Here's a straightforward question I can't seem to find a good answer to. Let's say you're using some variables to predict age. I'm assuming a regression model is the right approach. In this case, what would be a suitable metric to evaluate the model performance? My sense is that MAE and RMSE aren't appropriate because they assume your predictions can be equally wrong in both directions, but that's not true: predictions can be larger in the positive direction than in the negative direction. For example, if the true age is 5, the prediction can be no less than 0, but could be infinitely high.

Is RMSE/MAE an appropriate metric to evaluate the size of the error, and if not, what would be? Or if there isn't an appropriate "size" metric, should I use R^2 or some other "fit" metric?

Answer 1

Either RMSE or MAE will give you a sense of accuracy in prediction. The exact value of any error metric is up to your specific use case and tolerance.

In other words, if you are predicting ages of 5-10 year olds and see a RMSE of 5 then you might think that you have a significant problem. But if you are predicting ages expecting them to be 40-80 year olds, then a RMSE of 5 wouldn't be a big concern right?