2
$\begingroup$

I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.

I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.

The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?

$\endgroup$
  • $\begingroup$ What do you mean by "splitting based on participant?" $\endgroup$ – Ethan Mar 18 '19 at 17:27
  • $\begingroup$ @Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data. $\endgroup$ – Andrew Mar 18 '19 at 20:23
  • $\begingroup$ I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself. $\endgroup$ – Simon Larsson Mar 19 '19 at 12:54
2
$\begingroup$

You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

where groups is an array of group indices.

$\endgroup$
0
$\begingroup$
id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
    if i in train_split:
#         print("Training: ", (i+1))
        blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
        blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
    else:
#         print("Testing: ", (i+1))
        blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
        blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.