I have read a paper on Negative Learning: https://arxiv.org/abs/1908.07387. The idea is that you can train a network not only by telling what label of the sample is, but by telling what it surely is not. Let's call the latter a "negative" label.

An excerpt from the paper says (top formula is for usual "positive" label loss (PL), bottom - for "negative" label loss (NL):

An excerpt from the paper

I have a problem where "negative" labels collection is much easier than labeling each sample. So it is tempting to use it.

Is there some implementation of such a loss function in pytorch? Or should I write a custom loss layer code? If so, how should I do it?

  • 2
    $\begingroup$ It seems that you would have to create a custom loss function for this, which can be done relatively easily in pytorch by either creating a custom python function or using the nn.Module class (see also this webpage). $\endgroup$
    – Oxbowerce
    Apr 16, 2021 at 13:18

1 Answer 1


One implementation is

( (loss+loss_neg) / (float((labels>=0).sum())+float((labels_neg[:,0]>=0).sum())) ).backward()

from NLNL-Negative-Learning-for-Noisy-Labels GitHub repo.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.