Say I have multiple domains such that
d_i is drawn from
D=[d_1, d_2, ... d_K]. We have two options to train a CNN which equally represents all domains.
- Collect samples from all domains and create a batch B
- Generate a batch B from a randomly chosen
d_iand keep shuffling domains per iteration
My intuition tells me that option A is better. However, I do not know why. I am looking for research which touches upon this topic. I would also appreciate your intuition, choice of normalization layers and other tricks/techniques that helped you navigate this problem.