Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Which loss function is the best loss function when using XGB regression with a highly skewed dataset?
The skewness of the data is very high. I used XGBoost with objective function of linear regression (but the data was transformed into the log space). It performed better than using gamma objective function. Any other suggestions?
You could try weighting the loss for each data point, such that some datapoints do not dominate the loss.
Since this might be a question for other people, here are the results of my findings:
I have tried two options using XGB Regression with different objective functions including:
A linear regression objective function ("reg:linear" or "reg:squarederror") and transformed the target to the log space
A gamma objective function ("reg:gamma"), which is useful for a skewed target with gamma distribution, e.g., insurance claims severity. In this case, I did not transform my target to the log space.
In my case, option 1 performed better than option 2 (around 15-20%). However, depending on the nature of the data one of them might outperform the other one.
Also, here is one potential option as the objective function "reg:squaredlogerror"
