How does one learn a classifier from data that isn't always fully labelled? For example, say one has corrupted data from the CIFAR-10 dataset (which has labels like bird/automobile/ship/truck). Now this corrupted data (X, Y) pairs and preserves X, while "confusing" a large number of Y pairs by replacing each label with a set of labels the sample's true label is from. So a label "bird" may become "not ship", "automobile" may become "autombile or truck", "ship" may become "ship" (unchanged) etc.

How does one best exploit this information? Is there a loss function that handles these?


1 Answer 1


If the amount of incorrect labels isn't big, in complaisant to correct ones, you can still train your model as usual and then plot a confusion matrix for different validation sets, which can tell you how many classifications were incorrect and which of them. After that you can decide to either correct the mistakes or dump them from the dataset

Example using fastai and pet breed image classifier (numbers outside of the main diagonal are mistakes):

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix(figsize=(12,12), dpi=60)

confusion matrix

# see the top errors
  • $\begingroup$ This makes sense. But in my problem the corrupted points are not insignificant. I've updated my question to reflect this better. Thanks! $\endgroup$ May 26, 2023 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.