Is 25-75 a good case-control ratio for a neural network?

Question

I have 50,000 DNA samples from a biobank. Only 2,200 samples have the label (disease) that I want to work with for supervised learning in a neural network.

So my question is, how many of those non-diseased samples do I need to include in my training data? If the diseased samples are my cases, how big should my control group be? Twice as big? The same size? What would a good ratio be?

The feature space is insanely big, so I am worried about training times. It feels unnecessary to include all the other 48,000 controls.

And yes, yes I know it will depend on the accuracy that I am seeing in the model, but what’s a good starting ratio?

Answer 1

I don't think there really is an optimal answer to a question like this. I would advise that the control group should be at least 50%, but anything more than that would be better. For a NN model the more examples you give it to train on, the more patterns it will learn and the more accurate it will be. If you could use the entire set, then do!

It's really down to time constraints and processing power. Try starting by using a small sample, maybe a few dozen or a few hundred samples and see how long that takes. Then try a sample roughly twice the size and check it again. You can then extrapolate the size of the sample you could use for given time constraints.

If you use this approach make sure to use a stratified k-fold split for cross validation. That will keep the relative distributions in your samples. Hope this helps.