I created a neural network with multi-label classification using MSE. Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the two targets like this?

target = t * target1 + (1-t)* target2

2 Answers 2


In their paper, mixup: Beyong Empirical Risk Mininization, the authors provide a piece of code how to train an epoch in PyTorch:

# y1, y2 should be one-hot vectors
for (x1, y1), (x2, y2) in zip(loader1, loader2):
    lam = numpy.random.beta(alpha, alpha)
    x = Variable(lam * x1 + (1. - lam) * x2)
    y = Variable(lam * y1 + (1. - lam) * y2)
    loss(net(x), y).backward()

As you can see, they only use a single loss function defined on the new combined target variable y or, in your terminology, target.

  • $\begingroup$ Thank you. However, I am not sure if this also holds for multi-label classification using MSE. In the code referenced by you there is a comment "y1, y2 should be one-hot vectors". $\endgroup$
    – codeprof
    Commented May 28, 2021 at 9:02

For the question "Does Mixup require two loss functions", the answer is no. You can use the same loss function that you would have used without Mixup augmentation.

For the question of which loss function to use when each sample can have more than one class, the following discussion may be helpful:



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.