0
$\begingroup$

The Setting

  • Let's suppose that I have an imbalanced dataset.
  • For training purposes, I want to implement a dataloading scheme that samples from this dataset in a more balanced way.
  • I want to leverage existing metadata for this purpose.
  • Each instance in my dataset belongs to either category $A$ or category $B$. Similarly, each category can be subdivided into several subcategories, namely, $A_1$, $A_2$, $A_3$, $A_4$, ..., $A_N$ and $B_1$, $B_2$, $B_3$, $B_4$, ..., $B_M$.

How I want the dataloading to work

My goal is to have the model learn to discriminate between $A$ and $B$ (top priority) and also between the subcategories within each category.

So I had the following in mind:

  • I would like to have a separate dataset for each subcategory $A_1$, $A_2$, $A_3$, $A_4$, ..., $A_N$ and $B_1$, $B_2$, $B_3$, $B_4$, ..., $B_M$.
  • Then I would like to have two "meta-datasets", a meta-dataset A as a wrapper for $A_1$, $A_2$, $A_3$, $A_4$, ..., $A_N$, and a meta-dataset B as a wrapper for $B_1$, $B_2$, $B_3$, $B_4$, ..., $B_M$. A meta-dataset should be able to sample from the sub-datasets it contains in a more balanced way following some heuristic. Question: Is this possible? How can I do this?
  • Finally, I would like to have a single "meta-meta-dataset" as a wrapper for meta-dataset $A$ and meta-dataset $B$. This meta-meta-dataset should be able to sample in a balanced way from meta-dataset $A$ and meta-dataset $B$. Question: Is this possible? How can I do this?

In other words, during my training loop, I want all my batches to be relatively balanced in terms of categories $A$ and $B$, and within each category I would like the subcategories to be sampled more or less uniformly as well.

Does anyone know how something like this can be done with very imbalanced datasets in Pytorch?

Note: Keep in mind that it's very likely that I won't be able to fit all subcategories into a single batch (there may be many subcategories and the GPU might not be big enough to sample from all subcategories at the same time), therefore it's okay if only a subset of subcategories of $A$ and a subset of subcategories of $B$ are sampled in each batch, as long as all subcategories are sampled more or less uniformly on average.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.