0
$\begingroup$

For regression module evaluation, I think only the MAE (Mean absolute error) value is not objective or practical.
Consider following situations:

  • A
    MAE=1 while target value follows the uniform distribution on the interval [1,100]
  • B
    MAE=1 while target value follows the uniform distribution on the interval [1,10]

Obviously A model is better.

So how to get an intuitive value for regression module evaluation, regardless the data set's target value's scope?

$\endgroup$
7
  • $\begingroup$ You described a metric of model precision to us. What kind of answer do you want? $\endgroup$
    – David Dale
    Nov 18, 2017 at 8:19
  • $\begingroup$ "Obviously B model is better" - not obvious to me at all. If I rescale the interval [1,100] to [1,10], the MAE is only approximately 0.1. I can't quickly see if comparing models on different datasets is meaningful. To me, for a fixed dataset, R^2 is just a way to see how much better a predictor is than the simple average. Similar for your value, except that for MAE the mean is not so natural mathematically, perhaps even worse than median. $\endgroup$
    – Valentas
    Nov 18, 2017 at 9:39
  • $\begingroup$ @DavidDale , To get a intuitive value, indicating how many times the model better than another model which always predicts mean value. $\endgroup$
    – yichudu
    Nov 20, 2017 at 2:22
  • $\begingroup$ @Valentas . Sorry, I was careless. I edit my post and give my own answer. Please make an remark. $\endgroup$
    – yichudu
    Nov 20, 2017 at 2:32
  • $\begingroup$ have you tried using MAPE instead of MAE? $\endgroup$
    – Toros91
    Nov 20, 2017 at 2:42

1 Answer 1

1
$\begingroup$

To get an intuitive contrast, I came up with:

Refer to $$R^2=1-\frac{\sum ( y_i - \hat y_i)^2} {\sum (y_i - \bar y)^2}$$

My method is: $$My Value=\frac {\sum |y_i - \bar y|} {\sum | y_i - \hat y_i|} $$ Indicating how many times the model better than other model, always predicts mean value.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.