Sample selection through clustering

Question

I have a biased set of samples going into a binary classification sklearn pipeline, white and black samples.

It is easy for me to fetch as many black samples as required, while whites are a bit harder to get. I suspect the black set is unbalanced and there for I have an incentive to preprocess as many black samples as my hardware permits, and pick the most representative samples.

The feature set (after dropping completely invariant features) is about 28k features.

The best approach I came up with is to cluster my black samples to a percentage of my white samples. For example say I have 500k white samples, I would then cluster the black samples to 100k clusters and pick five samples (randomly?) out of each cluster.

My questions are the following:

1. How can I have a transformer that will cluster the samples and mask out the unneeded samples in sklearn? Is there something ready for such tasks or should I build my own transformer based on the clustering algorithm?
2. Can you point at any potential issue with what I plan to do? I think one potential risk here is not clustering the white samples, but since I'm not getting enough of those anyway...
3. What clustering algorithms might be a good fit? I'm looking a different clustering algorithms that accept the number of clusters as parameter right now I plan to experiment with most and see which ones work best, but a direction will be appreciated.
4. Additionally, I'd like to read suggestions about other approaches to sample subsampling in my case.

Answer 1

Clustering will be much too expensive for your purpose (most are O(n^2), and the good ones like HAC may even be O(n^3) - you won't be able to run them on 300k instances). Also beware of the prerequisites of the various algorithms - they may not be applicable to your data, or the results may be as bad as random! Also, clustering algorithms may behave rather unpredictable, and will profuce unbalanced data sets. It would not be unusual if 99% of your data ends up in the same cluster, while other clusters are almost empty (or even empty!)

I would consider first to remove all duplicates and maybe near-duplicate records (ignoring identifiers, of course). These algorithms are much more scalable and tolerate more data types. That may already be enough to achieve your goal of data reduction.