I have a Pytorch regression model as follows:
model = nn.Sequential(nn.Linear(n_in, n_h_1),
nn.ReLU(),
nn.Dropout(p=0.1),
nn.Linear(n_h_1, n_h_2),
nn.ReLU(),
nn.Linear(n_h_2, n_out))
As you can see, I have used a Dropout regularization layer with dropout probability of 0.1.
When the training is completed, we should disable the dropout. This can be achieved using model.eval()
. Reference to this StackOverflow answer and other resources, we should multiply the output of hidden layer with (1-p)
during inferencing of model.
My question is: do we have to do this manually using pytorch or the library handles this itself after applying model.eval()
?
If we have to do it ourselves, how can we achieve thisfor the given model?