I have a dataset of audio and text files that I want to balance using different criteria to train a neural network. The text and matching audio file are grouped under one ID.
For each ID, I have a number of words and some metadata.
Let's say my data have three metadata/criteria that need to be balanced (well... kind of, I want to be able to choose the percentage for some criteria values) :
- The name of the speaker, each speaker should have approximately the same number of words.
- The status of the speaker (active if the speaker is still in activity or inactive if not), where I want approximately 70 % active speakers and 30 % inactive.
- The gender of the speaker, where I want approximately 50 % male and 50 % female.
Example : ID12345 with 10,000 words. Metadata : John, male and active
If I balance one of the criteria, then another, I will surely unbalance the previously balanced one. Moreover, if I balance using only one criterion at a time, I could throw away data that could be useful to the balance of the following criteria, which mean I will end up with less data that I could have had optimally.
Is there an algorithm that could balance the data using all criteria with their percentage target, and maximise the number of words remaining ?
I'm looking for a general algorithm, with a variable number of criteria.
EDIT : I use C++ and MongoDB (but I'll accept other languages/tools if it can help me, as well as algorithms).