3
$\begingroup$

I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.

I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.

The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?

$\endgroup$
3
  • $\begingroup$ What do you mean by "splitting based on participant?" $\endgroup$
    – Ethan
    Mar 18, 2019 at 17:27
  • $\begingroup$ @Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data. $\endgroup$
    – Andrew
    Mar 18, 2019 at 20:23
  • $\begingroup$ I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself. $\endgroup$ Mar 19, 2019 at 12:54

2 Answers 2

2
$\begingroup$

You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

where groups is an array of group indices.

$\endgroup$
0
$\begingroup$
id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
    if i in train_split:
#         print("Training: ", (i+1))
        blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
        blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
    else:
#         print("Testing: ", (i+1))
        blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
        blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.