1
$\begingroup$

I'm beginning my data science journey and I've faced a challenge that confuses me a bit. I have a set with few features and a target variable whose raw distribution is highly skewed.

Raw y

I've read that it's possible to use a log transformation to normalize the target variable (loss in $) and thus increase the accuracy.

Logged y

When I train my model with "y_raw", using MAE I get an error of 306k. When I log-transform y = y.transform(np.log) I get MAE accuracy of around 2 (log-transformed units I suppose?), which is e^2 = 7.39 (y_raw). This is a significant drop from 306k to only 7.39 ($) (or am I getting it wrong?), so I am a bit suspicious about it.

So here are my questions: 1) Did I get it correct that the error rate drop from 306k to only 7.39 is real and is valid? 2) How do I make a predictions from there? If I feed a sample to my model, receive a log-transformed output, lets say it returned a prediction of y_log = 10. Do I then simply use an inverse of it by placing e^10 = 22,026.5 and will it be my final prediction?

Improve this question
$\endgroup$
2
$\begingroup$

Taking the log doesn't result in a normally-distributed target; it would tend to if the target was log-normally distributed, and you have something normalish there, not quite. But, this distribution isn't actually what matters.

What taking the log does is change your model of how errors arise when fitting a regressor. You're now saying that the target values are $e^{P + \epsilon}$ where $P$ is your model's prediction and $\epsilon$ is Gaussian noise. Or: $e^P e^\epsilon$. That part directly interacts with the assumptions in your regressor.

So what you're finding is that on average the predictions are wrong by a factor of 7.39, not +/- $7.39.

What you really want to do is evaluate MAE on actual target values vs $e^P$ . You probably have a better model but not that much better.

Improve this answer
$\endgroup$
5
  • $\begingroup$ So what you're finding is that on average the predictions are wrong by a factor of 7.39.- How can I easily transform it to the "raw" numbers to estimate the error rate in terms of $? By using my log-MAE result? $\endgroup$ – Anonymous User Jun 7 '20 at 16:17
  • 1
    $\begingroup$ You need to exponentiate your predictions to get them in dollars, then compare the absolute differences from actual values. $\endgroup$ – Sean Owen Jun 7 '20 at 23:18
  • $\begingroup$ I'm a little bit confused now. By using the obtained error coefficient, namely 2, isn't e^2 = 7.39 $ follows the way you described? $\endgroup$ – Anonymous User Jun 8 '20 at 11:54
  • $\begingroup$ If your log-space prediction is $P$ and error is $\epsilon$ in log space, then your prediction in dollars is $e^P$ but the error in dollars is not $e^\epsilon$. It's $e^{P+\epsilon} - e^P = e^P(e^\epsilon - 1)$. It's multiplicative in dollars. $\endgroup$ – Sean Owen Jun 8 '20 at 14:55
  • $\begingroup$ Got it! It is clear now. Thank you! $\endgroup$ – Anonymous User Jun 9 '20 at 10:40
1
$\begingroup$

Of course your error rate is going to decrease. Remember that your changes in MAE values may come from the fact that the scales of your original variable and that variable transformed by a logarithm ARE NOT THE SAME, and mean is scale-dependant.

About your second question, is exactly that!

If you would like to compare the use (or not) of logarithms. You could use your original model and, when calculating MAE, apply logarithms to predicted and real values. Then, you will be able to compare the two models in terms of MAE.

Check that before the application of the logarithm you had several values close to 0, almost surely your model learned to output values close to 0 with an acceptable accuracy. Once you apply the logarithm, you allow your data to be more spread (differences are "easier" to see).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.