2
$\begingroup$

does anyone have any advice on how to implement this loss in order to use it with a convolutional neural network? Also, how should I encode the labels of my training data? We were using one hot encoding with bce loss before and I was wandering if I should keep it that way also for the hinge loss, since the label itself is not used in the formula of the loss other than for indicating which one is the true class. The dataset is CIFAR100 by the way. Thank you in advance!

EDIT: I implemented a version of this loss, the problem is that after the first epoch the loss is always zero and so the training doesn't go further. Here is the code:

class MultiClassSquaredHingeLoss(nn.Module):
    def __init__(self):
        super(MultiClassSquaredHingeLoss, self).__init__()

    def forward(self, output, y): #output: batchsize*n_class
        n_class = y.size(1)
        #margin = 1 
        margin = 1
        #isolate the score for the true class
        y_out = torch.sum(torch.mul(output, y)).cuda()
        output_y = torch.mul(torch.ones(n_class).cuda(), y_out).cuda()
        #create an opposite to the one hot encoded tensor
        anti_y = torch.ones(n_class).cuda() - y.cuda()
        
        loss = output.cuda() - output_y.cuda() + margin
        loss = loss.cuda()
        #remove the element of the loss corresponding to the true class
        loss = torch.mul(loss.cuda(), anti_y.cuda()).cuda()
        #max(0,_)
        loss = torch.max(loss.cuda(), torch.zeros(n_class).cuda())
        #squared hinge loss
        loss = torch.pow(loss, 2).cuda()
        #sum up
        loss = torch.sum(loss).cuda()
        loss = loss / n_class        
        
        return loss
$\endgroup$
1
$\begingroup$

One option is to use the existing torch.nn.MultiMarginLoss. For squared loss, set p=2.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.