I have been reading "drop" is a method to regularize model better. It's purpose is to update only some % of weights in backprop and it helps you to not over fit the model. But I am wondering, is this method useful in minibatch at all?
1 Answer
$\begingroup$
$\endgroup$
Yes dropout is useful this case too. The purpose of dropout is to make the model generalize better by causing it to use all available weights from a given layer. It is supposed to make the subsequent layer not rely on a small number of activations from the previous layer, but to use them all.