In the PyTorch nn
module there are 2 types of dropouts:
- A normal Dropout - During training, randomly zeroes some of the elements of the input tensor with probability p using samples from a Bernoulli distribution. Each channel will be zeroed out independently on every forward call.
A Dropout2d - Randomly zero out entire channels. Each channel will be zeroed out independently on every forward call with probability p using samples from a Bernoulli distribution.
- From Documentation
My question is when we use CNN, which of the dropout we use in the Convolutional layers. In this paper it is being said we use the first kind. But I am not so sure what is the general practice.