2
$\begingroup$

I'm currently tackling a regression problem with skewed target variable (presented below). enter image description here

Naturally, my first idea was to transform the target with natural logarithm as it'll probably help both linear regression or decision-tree-based algorithms. The second idea is to prepare a validation scheme similar to stratified k-fold cross-validation with target binned into n groups. However, my concern is that I have only few highest values:

enter image description here

Therefore, my test set and all validation sets error are highly dependent if one of these 4 extreme values are drawed placed within them or not. That makes it hard to obtain reliable true error estimate.

Is there anything more I can do to handle that issue?

$\endgroup$
1
$\begingroup$

You can generate symmetrical distribution(s) by suitable transformation: Your distribution is mainly right-skewed, therefore a log10 transformation is required.

You can also use an auto-binning method and combine it with dummy variables for the spikes.

If you then perform a feature selection, the learner will automatically choose the most significant features. Learners like XGBoost automatically take care of multicollinearity.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.