does anyone have any advice on how to implement this loss in order to use it with a convolutional neural network? Also, how should I encode the labels of my training data? We were using one hot encoding with bce loss before and I was wandering if I should keep it that way also for the hinge loss, since the label itself is not used in the formula of the loss other than for indicating which one is the true class. The dataset is CIFAR100 by the way. Thank you in advance!
EDIT: I implemented a version of this loss, the problem is that after the first epoch the loss is always zero and so the training doesn't go further. Here is the code:
class MultiClassSquaredHingeLoss(nn.Module): def __init__(self): super(MultiClassSquaredHingeLoss, self).__init__() def forward(self, output, y): #output: batchsize*n_class n_class = y.size(1) #margin = 1 margin = 1 #isolate the score for the true class y_out = torch.sum(torch.mul(output, y)).cuda() output_y = torch.mul(torch.ones(n_class).cuda(), y_out).cuda() #create an opposite to the one hot encoded tensor anti_y = torch.ones(n_class).cuda() - y.cuda() loss = output.cuda() - output_y.cuda() + margin loss = loss.cuda() #remove the element of the loss corresponding to the true class loss = torch.mul(loss.cuda(), anti_y.cuda()).cuda() #max(0,_) loss = torch.max(loss.cuda(), torch.zeros(n_class).cuda()) #squared hinge loss loss = torch.pow(loss, 2).cuda() #sum up loss = torch.sum(loss).cuda() loss = loss / n_class return loss