Let's say that I want to train a neural network to recognize symptoms of severe dehydration in distance runners visually. Runners tend to finish races either overhydrated (hyponatremic) or dehydrated, but the two present with fairly similar symptoms, so knowing the difference could be quite useful.
I've got myself a small dataset of a few thousand faces of dehydrated runners, a few thousand of overhydrated runners, and several million faces of the general population. I don't expect to be able to properly train a network to recognize dehydration with just a few thousand faces of the dehydrated, but I'd like to see if I can pretrain the network using the several million faces so it learns to distinguish facial details, then train it on the actual overhydrated/dehydrated faces. But if I take some other problem, like distinguishing men from women, and pretrain it on that, I'd imagine my hydration training phase would output something that distinguishes something between hydration level and gender.
Is there some standard way to pretrain a model to learn general details (of faces for example) before training on the proper (far smaller) dataset?