3
$\begingroup$

Can anybody explain why/if target variable transformations could help when dealing with tree based models?

I've seen this excellent reply which explains quite well why it shouldn't affect if transforming inputs, but I haven't been able to find anything regarding outputs.

Can using a transformation like taking logs or using quantile transform of the response variable help?

Actually, we are using XGBoost and getting better results when using a normal quantile transform of our output and even better results when taking logs (our response variable is a price, highly skewed to the right) but I don't know if this is something justifiable by theory or just random chance.

Improve this question
$\endgroup$
4
  • $\begingroup$ What does better result mean in this case? How do you measure it? $\endgroup$
    – Viktor
    Commented Dec 17, 2018 at 23:12
  • $\begingroup$ In the sense of RMSE. Taking logs allows us to reach a smaller RMSE than we achieve when working with the untransformed target. $\endgroup$
    – Ludecan
    Commented Dec 17, 2018 at 23:43
  • 1
    $\begingroup$ Imagine you divide your target value by 10 and also your predictions by 10, what do you think how it will effect the RMSE (root mean square error)? RMSE is highly dependent on the scale you use. It is not a good measure to compare results on a different scale. So having different RMSE does not indicate that necessary that one model is better than the other. Try to transform back your predictions made on the log data to the original scale and compare the results that way. Do you still see difference when both data is on the same scale? $\endgroup$
    – Viktor
    Commented Dec 17, 2018 at 23:53
  • $\begingroup$ Yes, actually those are our tests, we backtransformed the predictions of the log model using exp. We trained a model directly on our target variable y. Calc RMSE for that model, say RMSE_1. Then trained another model on log(y) and to predict the original y, transform the raw model predictions using exp. Calc RMSE on those and get RMSE_2. Then RMSE_2 < RMSE_1. $\endgroup$
    – Ludecan
    Commented Dec 18, 2018 at 0:03

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.