1
$\begingroup$

I am new to ML. I am trying to create a training dataset that is equally distributed between multiple lists, each of which have a different kind of data. How can I do this? I looked into GroupKMeansFold, and StratifiedFold but I don't fully understand it.

Here's more information:

  1. I have three lists a, b, c.
  a = [a1, a2, a3.... a10000]
  b = [b1, b2, b3.... b10000]
  c = [c1, c2, c3.... c10000]

I want my training, test, and val data to have 70, 15, 15 splits. I also want the 70% of the training data to be equally distributed between lists a,b,c. Same with the test and val data.

I want the training data to have 70% of the data from a, b, and c. I want the val data to have 15% of the data from a,b, and c. I want the test data to have 15% of the data from a,b, and c

$\endgroup$
4
  • 1
    $\begingroup$ Question is not clear. Can you elaborate on what do you mean by equally distributed between lists? Give a small sample of how your output should look like. $\endgroup$ Jul 29 at 12:48
  • $\begingroup$ Maybe it would be best to explain why you need this for? It looks like you're artificially creating a dataset, but this is not recommended for realistic applications. $\endgroup$
    – Erwan
    Jul 30 at 10:47
  • $\begingroup$ @Erwan, the lists have sample data. I also don't see why generating synthetic data for a real application is a bad idea, but that's for another conversation. :) $\endgroup$
    – user81371
    Jul 30 at 15:34
  • $\begingroup$ @user81371 I'm saying this because it's extremely rare for real data to be perfectly balanced. I don't know what the 3 lists represent, but it's unlikely that whatever they represent comes with a uniform distribution (?). If they don't, the validation and test sets that you generated are not representative, this can be a serious evaluation issue. $\endgroup$
    – Erwan
    Jul 30 at 16:01

1 Answer 1

1
$\begingroup$

Imho it's possible that you're overthinking this:

Easy option: regular sampling

concatenate all three lists together, then shuffle the dataset and then take the first 70% elements as training set, the next 15% as validation set and the last 15% as test set.

This option is probably enough for your purposes, but the global random sampling means that the number of items from every list is approximate. For example the training set may have 6981 items from list a instead of 7000.

Serious option: stratified sampling

To be extremely precise about the proportions across lists, what you need is stratified sampling. Stratified sampling consists in sampling independently for every value of the target variable, in your case the list. This means for example:

  • Pick 7000 items from a
  • Pick 7000 items from b
  • Pick 7000 items from c

The training set is the concatenation of the 21,000 items. Same process for validation and test set.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.