I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always split the data into train and test set before doing any kind of resampling to avoid info leakage but how what strategy gives better results - random train_test_split
or StratifiedShuffleSplit
? The general sentiment is that the latter seems to be outperforming the former in case of an imbalanced data set. Btw, how does StratifiedShuffleSplit
work?
Also, how to factor in the cross-validation set and how to process it?
Should it also contain the same number of positive and negative samples as per the sampling was done on the training set?