2
$\begingroup$

I have a dataset of audio and text files that I want to balance using different criteria to train a neural network. The text and matching audio file are grouped under one ID.

For each ID, I have a number of words and some metadata.

Let's say my data have three metadata/criteria that need to be balanced (well... kind of, I want to be able to choose the percentage for some criteria values) :

  • The name of the speaker, each speaker should have approximately the same number of words.
  • The status of the speaker (active if the speaker is still in activity or inactive if not), where I want approximately 70 % active speakers and 30 % inactive.
  • The gender of the speaker, where I want approximately 50 % male and 50 % female.

Example : ID12345 with 10,000 words. Metadata : John, male and active

If I balance one of the criteria, then another, I will surely unbalance the previously balanced one. Moreover, if I balance using only one criterion at a time, I could throw away data that could be useful to the balance of the following criteria, which mean I will end up with less data that I could have had optimally.

Is there an algorithm that could balance the data using all criteria with their percentage target, and maximise the number of words remaining ?

I'm looking for a general algorithm, with a variable number of criteria.

EDIT : I use C++ and MongoDB (but I'll accept other languages/tools if it can help me, as well as algorithms).

$\endgroup$
0
$\begingroup$

You would need to balance the data according to the joint distribution of all 3 features.

One simple option is to treat each feature as categorical (binning any continuous features), then map each combination of features to a single new feature:

doc_len_tokens  | status   | gender | level
----------------|----------|--------|------
1               | active   | male   | 1
1               | active   | female | 2
1               | inactive | male   | 3
1               | inactive | female | 4
2               | active   | male   | 5
2               | active   | female | 6
2               | inactive | male   | 7
2               | inactive | female | 8
... etc ...

You would then stratify on level when sampling.

Here is an example in SQL:

SELECT
    data.unique_id,
    data.doc_num_tokens,
    data.status,
    data.gender
FROM (
    SELECT
        data.unique_id,
        data.doc_num_tokens,
        data.status,
        data.gender,
        row_number() OVER (
            PARTITION BY
                data.doc_num_tokens,
                data.status,
                data.gender
            ORDER BY
                newid()
        ) AS row_num
    FROM
        my_data AS data
) AS data
WHERE
    row_num <= ${number of rows to sample}
$\endgroup$
  • $\begingroup$ Thanks, but there are some problems with this : • It doesn't take into account the number of words. If John who is an active male have texts that are longer than others (for example), I'll get a lot more of words for the level corresponding to <John, active, male>. • I can't select target percentages for each value of criteria. For example, I'll get 50 % active and 50 % inactive, but I want 70 % and 30 %. Maybe I'm missing something, I'm not accustomed with Pandas. I'm using C++ and MongoDB. $\endgroup$ – Adrian B. Oct 10 '18 at 21:20
  • $\begingroup$ @AdrianB. the name_tokens_len feature is meant to represent number of words. You stratify according to the level feature I constructed on the right side. It's just a number that corresponds to a unique combination of the other features. $\endgroup$ – shadowtalker Oct 10 '18 at 22:03
  • $\begingroup$ @AdrianB. I converted it to a SQL example, which is more or less the same thing $\endgroup$ – shadowtalker Oct 10 '18 at 23:34
  • $\begingroup$ Unfortunately, the possibility to choose a target percentage for some of the criteria values (like 70 % active and 30 % inactive) is something I need and that isn't compatible with your response. Just by curiosity, how did you obtain name_tokens_len from the number of docs ? I think I understood the rest of your idea, except this part. $\endgroup$ – Adrian B. Oct 11 '18 at 15:32
  • $\begingroup$ @AdrianB. I thought you meant "same number of words in the speaker's name". I also misunderstood that you want a specific distribution of individuals within your sample. For that you can maybe use some kind of rejection sampling, although there is probably a more efficient scheme. $\endgroup$ – shadowtalker Oct 11 '18 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.