If you take the following sentence from an article on deep neural networks
to regularize the classifier layer by estimating the marginalized effect of label-dropout during training.
What does label-dropout mean
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ For understanding that, you must understand what a drop out mean. This would help you with that :) $\endgroup$– Dawny33Commented Feb 21, 2016 at 15:51
-
$\begingroup$ Care to help explain drop out $\endgroup$– MichaelCommented Feb 22, 2016 at 19:06
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I ended up finding the paper you were referencing, maybe next time, add it to your post.
From what I can understand in this paper, label-dropout means that you are dropping the real labels and replace them with others.
The section of the paper you're referring to is explaining everything in great detail, so I'd try to read it carefully. In short, instead of using the ground-truth distribution (i.e. one 1 and a bunch of 0s) in the target layer, they are adding some uncertainty. That way, the model becomes less confident, and therefore is less prone to overfitting and becomes more adaptable.
answered Oct 15, 2017 at 13:51
$\begingroup$
$\endgroup$
1
Perhaps it means groundtruth just keep one correct label while dropping out others. And this can cause some side-effects under extreme circumstances.
-
$\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$– Community BotCommented Feb 6, 2022 at 17:47