Let's assume we have a large annotated dataset with 4 classes. In this dataset, there might be annotated images with less than 4 classes, where the remaining classes might or might not be present. As an example, say we want to detect pedestrians/cars/bicycles/roads in images. In our dataset, there are some annotated images with only 3 classes: pedestrians/cars/bicycles, but this does not mean there are no roads in these images. That is, there might be roads in these images that were ignored by the annotator for some reason, or there might no be roads at all. My question is, how do we take this uncertainty into the loss term?
One option is to work with independent networks for each class. But what if we want to train a single network? how do we add something like a "don't care" term for objects not annotated in an image, but might still be present in the image?