How do I bootstrap a Dataset/DataFrame with multiple continuous and categorical columns?
For eg: Say I'm trying to bootstrap the colour distribution of M&M's and I have 50 bags (samples) each with recorded counts of 6 colours, but the samples/bags randomly come from two factories, and have 2 different time stamps on the sample/bag packets.
What would be a good method of bootstrapping this kind of dataset? Using loops that resample slices by categorical values? eg: Split the dataset into serial_no groups E2 and E1, resample those with replacement and combine, then do the same for serial_time etc...
Or can the whole dataset be resampled at once? If there happens to be subtle patterns within 'E2' data for eg, how is that preserved in bootstrapping?
# create dataframes for serial_no columns. E1_bs = pd.DataFrame() E2_bs = pd.DataFrame() # proportinal sample with replacement E1_df = df[df['serial_no'] == 'E2'] sample_size = len(E1_df)*10 for i in range(len(E1_df)): E1_swr = pd.DataFrame(E2_df.sample(sample_size, replace=True)) E1_bs = pd.concat([E1_bs,E1_swr],axis=0) E2_df = df[df[['serial_no'] == 'E1'] sample_size = len(E2_df)*10 for i in range(len(E2_df)): E2_swr = pd.DataFrame(E2_df.sample(sample_size, replace=True)) E2_bs = pd.concat([E2_bs,E2_swr],axis=0) bootstrap = pd.concat([E2_bs,E1_bs],axis=0)
If I do this for all of the categorical columns (serial_no and serial_time in this case) and add them together as one dataset, I get different means (significantly) compared to if I just sample the whole thing at once using the same total amount of samples).
# Compared to bootstrapping the whole thing sample_with_replacement = pd.DataFrame(df.sample(len(bootstrap), replace=True))