2
$\begingroup$

Which loss function is the best loss function when using XGB regression with a highly skewed dataset?

The skewness of the data is very high. I used XGBoost with objective function of linear regression (but the data was transformed into the log space). It performed better than using gamma objective function. Any other suggestions?

$\endgroup$
4
  • $\begingroup$ Did you figure out a good solution for this? I asked a similar question this morning: datascience.stackexchange.com/questions/75903/… $\endgroup$
    – gammapoint
    Jun 12, 2020 at 15:56
  • 1
    $\begingroup$ I post my response here and also on your post. $\endgroup$
    – nimar
    Jun 12, 2020 at 17:07
  • $\begingroup$ Sorry, but the dataset I’m using contains some proprietary and confidential information. $\endgroup$
    – gammapoint
    Jun 12, 2020 at 18:20
  • $\begingroup$ The data are skewed, or the assumed conditional distribution (which you don’t observe) is skewed? Those aren’t the same. $\endgroup$
    – Dave
    Oct 23, 2023 at 23:33

2 Answers 2

0
$\begingroup$

You could try weighting the loss for each data point, such that some datapoints do not dominate the loss.

$\endgroup$
0
$\begingroup$

Since this might be a question for other people, here are the results of my findings:

I have tried two options using XGB Regression with different objective functions including:

  1. A linear regression objective function ("reg:linear" or "reg:squarederror") and transformed the target to the log space

  2. A gamma objective function ("reg:gamma"), which is useful for a skewed target with gamma distribution, e.g., insurance claims severity. In this case, I did not transform my target to the log space.

    In my case, option 1 performed better than option 2 (around 15-20%). However, depending on the nature of the data one of them might outperform the other one.

Also, here is one potential option as the objective function "reg:squaredlogerror"

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.