Let's assume we have a large annotated dataset with 4 classes. In this dataset, there might be annotated images with less than 4 classes, where the remaining classes might or might not be present. As an example, say we want to detect pedestrians/cars/bicycles/roads in images. In our dataset, there are some annotated images with only 3 classes: pedestrians/cars/bicycles, but this does not mean there are no roads in these images. That is, there might be roads in these images that were ignored by the annotator for some reason, or there might no be roads at all. My question is, how do we take this uncertainty into the loss term?

One option is to work with independent networks for each class. But what if we want to train a single network? how do we add something like a "don't care" term for objects not annotated in an image, but might still be present in the image?


1 Answer 1


That problem is commonly called weak labels and is associated with weak supervision.

Weak labels are often not handled by the loss function. Most conventional loss functions assume perfectly labeled data points and have no mechanism to handle label uncertainty.

An alternative framework for weak labels is Confident Learning which

characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence.

  • $\begingroup$ Can you tell in what way you see the relation of weak labels to the problem of possibly missing annotations? My understanding is that weak labels refer to possibly incorrect or non-localized labels (the motivation is cheaper annotations). However my question is how to treat missing annotations in images (it is unknown whether they exist and where for these images) when working with multi-class semantic segmentation. $\endgroup$
    – r1c
    Mar 7, 2021 at 7:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.