0
$\begingroup$

I have read and seen that in CNN we apply DROPOUT layer between the FULLY CONNECTED layers to reduce overfitting.

Can we also apply the dropout layer between the CONV layers and the POOL layers. I have not seen models with this method applied. Will it help in overfitting when applied between these layers or are there any disadvantages to it?

$\endgroup$
1
2
$\begingroup$

In a CNN, each neuron produces one feature map. Since spatial dropout which is normally used for CNN's is per-neuron, dropping a neuron means that the corresponding feature map is dropped.

Pooling usually operates separately on each feature map, so it should not make any difference if you apply dropout before or after pooling.

Yes it should help with preventing overfitting. The added advantage is you can get uncertainty from the network at zero cost if using dropout.

$\endgroup$
4
  • $\begingroup$ By neuron do you mean a kernel filter? and do you mean dropout closes the output generated by that filter?? $\endgroup$ – Shiv Oct 29 '20 at 9:51
  • $\begingroup$ Yes, I do mean a kernel filter $\endgroup$ – AbinavR Oct 29 '20 at 9:52
  • $\begingroup$ Please can you explain your last line in detail ie. uncertainty from the network at zero cost. $\endgroup$ – Shiv Oct 29 '20 at 9:52
  • $\begingroup$ There is a method called MC dropout which arxiv.org/pdf/1506.02142.pdf explains. In this they add MC sampling at inference time which provides uncertainty. $\endgroup$ – AbinavR Oct 29 '20 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.