1
$\begingroup$

How does one learn a classifier from data that isn't always fully labelled? For example, say one has corrupted data from the CIFAR-10 dataset (which has labels like bird/automobile/ship/truck). Now this corrupted data (X, Y) pairs and preserves X, while "confusing" a large number of Y pairs by replacing each label with a set of labels the sample's true label is from. So a label "bird" may become "not ship", "automobile" may become "autombile or truck", "ship" may become "ship" (unchanged) etc.

How does one best exploit this information? Is there a loss function that handles these?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

If the amount of incorrect labels isn't big, in complaisant to correct ones, you can still train your model as usual and then plot a confusion matrix for different validation sets, which can tell you how many classifications were incorrect and which of them. After that you can decide to either correct the mistakes or dump them from the dataset

Example using fastai and pet breed image classifier (numbers outside of the main diagonal are mistakes):

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix(figsize=(12,12), dpi=60)

confusion matrix

# see the top errors
interp.plot_top_losses(5)
Improve this answer
$\endgroup$
1
  • $\begingroup$ This makes sense. But in my problem the corrupted points are not insignificant. I've updated my question to reflect this better. Thanks! $\endgroup$
    – mister_mole
    May 26 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.