I want to investigate the impact of various testing strategies on a product. Let's say chairs. I start with 500 random chairs that I've picked up from garage/yard sales. They come in all shapes and sizes, different manufacturers etc, but I've carefully measured each one and record: manufacturer, height, width, depth and fabric type. I calculate my population parameters. I find that some parameters are normal, and others are uniform, fabric types are mostly cotton but some leather.

I want to split my 500 chairs into groups of 100 such that each group has similar sample statistics. This way I can differentiate the impact of the various test on the chairs without worrying that I'm actually observing differences in my input distribution. I.e. I don't want all the leather chairs in one group.

I've tried randomly grouping my dataset, but I always end up with bad bias in one statistic. I thought that it might be possible to start with random grouping and then randomly selecting a pair of chairs to swap groups; recalculate group statistics; if they get closer to population parameter, keep the swap, else revert; repeat. That seems dreadfully slow.

I'm sure that there are many solutions available, but I'm not sure what they're called: what should I be searching for? If you have a solution handy, I don't really mind what language it is presented in. I'll add additional tags to help others find this sort of thing in the future. Thanks!


The term you are looking for is stratified sampling : https://en.wikipedia.org/wiki/Stratified_sampling. It's a way to sample from population that can be partitioned into sub-populations. More specifically for your problem, look at what they do for clinical trials, when they parition patients in sub-groups.

  • $\begingroup$ Thanks for your answer. I did end up looking into stratified sampling, which works very well for categorical data, but it still isn't clear to me what the best way is to ensure that the mean and sigma are the same for continuous variables across strata. My best solution is still randomly picking millions of samples until one matches the population statistics. $\endgroup$ – Marty Mar 25 '20 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.