0
$\begingroup$

It is usually better if you have a not so large but balanced dataset and you are performing classification to apply stratification in order to split it in a training and testing datasets which are both balanced as well.

So there is a notion of having the training and testing datasets represent adequately the overall dataset.

Could you expand this notion to regression?

So the idea is that instead of just shuffling and splitting the dataset, you could group the targets in bins for example [0, 10] [10, 20] etc. And then have the training and testing dataset be also an adequate representation of the whole dataset by having targets with all kinds of values. (Otherwise, you could end up leaving out some part of the range)

Makes sense? :)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Yes that makes sense. Your training and testing datasets should both be similar distributions to the whole dataset in both classification and regression, and in the regression case, binning the target variable is one way to achieve that. You need to make sure that you choose good bin sizes -- you can completely skew how the distribution looks in a histogram based on the bin sizes. Too small and you only have one or two examples per bin, resulting in a very erratic histogram. Too large and you lose a lot of information about the shape of the distribution.

Binning is just one way to approximate the density function of the distribution. Depending on how your data looks, you might be able to fit a Gaussian (or any other distribution) curve to your target variable and use that instead to sample your train/test split.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the response. Well, wouldn't you choose as many bins as possible? So if most bins would have for example as you say 2 samples, then the one would be the training set and the other would be the testing set. So you would have represented this target value to both datasets. No? $\endgroup$
    – George Pligoropoulos
    Commented Jun 3, 2017 at 13:01
  • 1
    $\begingroup$ @GeorgePligor If you want a 50/50 split for your training/test data, then I suppose the ideal case is when there are two examples per bin. Usually when I think of stratification, I think of something more along the lines of a 5 or 10-fold cross validation (80/20 or 90/10 splits) which is why I recommended a larger bin size. If you want a 50/50 split then by all means make the bins small! $\endgroup$
    – timleathart
    Commented Jun 5, 2017 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.