2
$\begingroup$

In data pre-processing, stratified shuffle is used to ensure that the distribution of the original dataset is reflected in the training, test and validation dataset.

Mini-batch gradient descent uses random shuffling to ensure randomness in the mini-batches.

My doubt is- Why should we implement stratified shuffle on our dataset if it is going to be shuffled in a random manner later during training?

$\endgroup$
2
$\begingroup$

It doesn't, the workflow when training a model is like that:

  1. Create 10 evenly distributed splits from the dataset using stratified shuffle
  2. train set = 8 splits; validation set = 1 split; test set = 1 split
  3. Shuffle the train set and the validation set and create minibatches from them
  4. Train for one epoch using the batches
  5. Repeat from step 3 until all epochs are over
  6. Evaluate the model using the test set

If we skip the stratified shuffling in step 1 the classes of the train set, validation set and test set wont be evenly distributed.

If we skip the shuffling before each epoch in step 3 the mini-batches in each epoch will be the same.

The proportions of the train set, validation set and test set can of course vary.

$\endgroup$
3
  • 1
    $\begingroup$ Correct me if I am wrong, based on your answer I deduce that stratification is done mainly to keep the distribution intact in validation as well as test dataset. It does not cause issues during training. $\endgroup$ Aug 9 '20 at 13:03
  • 1
    $\begingroup$ Yes, it doesn't cause issues during training. It's only done so the train set, validation set and test set are similary distributed. $\endgroup$ Aug 9 '20 at 17:25
  • 1
    $\begingroup$ All stratified StratifiedShuffleSplit does is shuffle the data randomly, then create x splits which have the same class distributions. So if your dataset has 80% rows from class A and 20% rows from class B it makes sure each of the x splits also has 80% rows from class A and 20% rows from class B. $\endgroup$ Aug 9 '20 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.