Is shuffling training data beneficial for machine learning?

Question

I was curious to know if shuffling ML training data is beneficial to better results?

Sorry not a lot of wisdom here, but I have been reading a post from pythonprogramming.net for this topic.

I copied this function from the post and modified to just save my shuffled data to csv file.

def Randomizing():
    df2 = df.reindex(np.random.permutation(df.index))
    df2.to_csv('C:\\Users\\Machine-Learning-Electric-Data\\randomized.csv')

Randomizing()

What appears to happen is only the index gets shuffled and all other data stays the same. I have many columns in my pd dataframe where I would need to keep all rows the same. (randomly shuffle all rows, its time series data) If this is beneficial can someone give me a tip on how to randomly shuffle my data more than just the index?

Answer 1

Shuffling the training data is generally good practice during the initial preprocessing steps.

When you do a normal train_test_split, where you'll have a 75% / 25% split, your split may overlook class order in the original data set. For example, class labels that might resemble a data set similar to the iris data set would include target variables that resemble the following:

For example: [0, 0, 0, 1, 2, 2, 2, 3, 3, 3, 3, 3]

You could see from this example above, that splitting your data without shuffling might lead to very poor performance in your test set evaluation. Said another way, you may only capture the classes 0, 1, and 2 in your training data and only 3 will be represented in your test data. Specifically for classification tasks, but also for other ML tasks it may be useful to shuffle your data. However, each situation is different so the best idea would be to try it both ways to see whether you see a significant improvement or not.

Hope this answers your question. Drop a comment if you would like any further clarification.