Does Mixup requires two loss functions?

Question

I created a neural network with multi-label classification using MSE. Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the two targets like this?

target = t * target1 + (1-t)* target2

Answer 1

In their paper, mixup: Beyong Empirical Risk Mininization, the authors provide a piece of code how to train an epoch in PyTorch:

# y1, y2 should be one-hot vectors
for (x1, y1), (x2, y2) in zip(loader1, loader2):
    lam = numpy.random.beta(alpha, alpha)
    x = Variable(lam * x1 + (1. - lam) * x2)
    y = Variable(lam * y1 + (1. - lam) * y2)
    optimizer.zero_grad()
    loss(net(x), y).backward()
    optimizer.step()

As you can see, they only use a single loss function defined on the new combined target variable y or, in your terminology, target.

Answer 2

For the question "Does Mixup require two loss functions", the answer is no. You can use the same loss function that you would have used without Mixup augmentation.

For the question of which loss function to use when each sample can have more than one class, the following discussion may be helpful:

https://stats.stackexchange.com/questions/207794/what-loss-function-for-multi-class-multi-label-classification-tasks-in-neural-n