I have a dataset that is bigger than I need it to be. In fact, bigger than my hardware can handle. So I'm trying to lower the number of samples. And I'm not sure what is the right approach to do so. Here are some other facts:
- I'll be applying some clustering algorithm afterward
- The outliers are the important data points in the dataset (there could be 1 outlier per each 100k records)
- There are 13 columns in this dataset each with a different distribution
I understand that if I wanted to keep the current distribution of the data, I should resample uniformly. But the fact is that the clustering algorithm does not benefit from having many close (almost identical) data points. So I would rather resample in a way that the resulting dataset is relatively uniform. But I'm not sure if this is a valid approach and if it is, how to do it (for instance in Python with Pandas).