0
$\begingroup$

I have read and seen that in CNN we apply DROPOUT layer between the FULLY CONNECTED layers to reduce overfitting.

Can we also apply the dropout layer between the CONV layers and the POOL layers. I have not seen models with this method applied. Will it help in overfitting when applied between these layers or are there any disadvantages to it?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
2
$\begingroup$

In a CNN, each neuron produces one feature map. Since spatial dropout which is normally used for CNN's is per-neuron, dropping a neuron means that the corresponding feature map is dropped.

Pooling usually operates separately on each feature map, so it should not make any difference if you apply dropout before or after pooling.

Yes it should help with preventing overfitting. The added advantage is you can get uncertainty from the network at zero cost if using dropout.

Improve this answer
$\endgroup$
4
  • $\begingroup$ By neuron do you mean a kernel filter? and do you mean dropout closes the output generated by that filter?? $\endgroup$
    – Shiv
    Commented Oct 29, 2020 at 9:51
  • $\begingroup$ Yes, I do mean a kernel filter $\endgroup$
    – AbinavR
    Commented Oct 29, 2020 at 9:52
  • $\begingroup$ Please can you explain your last line in detail ie. uncertainty from the network at zero cost. $\endgroup$
    – Shiv
    Commented Oct 29, 2020 at 9:52
  • $\begingroup$ There is a method called MC dropout which arxiv.org/pdf/1506.02142.pdf explains. In this they add MC sampling at inference time which provides uncertainty. $\endgroup$
    – AbinavR
    Commented Oct 29, 2020 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.