Questions tagged [dropout]

Dropout is a technique to reduce overfitting during the training phase of a neural network.

13 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2
votes
0answers
24 views

Intuitive reasoning behind inverted dropout in neural networks

I'm going through the deeplearning.ai course on Coursera and am trying to understand the intuitive reasoning behind inverted dropout in neural networks. Based on the lecture, my understanding is as ...
2
votes
0answers
491 views

Measuring uncertainty in an LSTM network using dropout in keras/tensorflow

I've created a simple LSTM network for testing ...
1
vote
0answers
29 views

What is dropout in convolutional layers and how does that different from max-pooling-dropout?

When dropout is applied to fully connected layers some nodes will be randomly set to 0. It is unclear to me how dropout work with convolutional layers. If dropout is applied before the convolutions, ...
1
vote
0answers
39 views

GAN training the average of my train data

I have been training a GAN with 1D convolutional layers on sinus functions. However if I start varying my sinus (random amplitude for example), the model generates only the average of the random range....
1
vote
0answers
62 views

Is applying simultaneous K Fold Cross Validation and Drop out possible?

Well, it might seem ridiculous but I was just thinking whether it is possible to have these two methods simultaneously or not. I ran the code and faced an error, but in theory it doesn't seem ...
1
vote
0answers
22 views

Dropout onto pre-weighted vs onto pre-activated vector?

For any layer in my neural net, should I apply dropout onto an entering vector, or on the pre-activated vector? In other words: $$\vec q=W\cdot \vec x$$ $$\vec h = activate(drop(\vec q))$$ or: $$\...
1
vote
0answers
297 views

Concrete Dropout for Recurrent Neural Networks (Keras)

I would like to use the Concrete Dropout Framework from GAL in application to recurrent neural networks. There is a great paper about it and the implementation can be found on the website (Thank you ...
1
vote
0answers
116 views

Why isn't Maxout used in the state of the art models?

I have just read the paper from Ian Goodfellow et al. titled "Maxout Networks". It seems that the Maxout activation should be quite powerful, as it can approximate any convex function, i.e. Relu, ...
0
votes
0answers
32 views

Parallel programming in Python

I have the next code that I am trying to run in parallel: ...
0
votes
0answers
12 views

Prove Ridge regression duality of dropout

EDIT: Found one mistake, confused rows and columns in the step of summing over column operations. I'm trying to prove the Ridge regression duality of Dropout, as described in section 9.1 of this paper....
0
votes
0answers
11 views

Is the first Linear layer in Pytorch considered as the input layer?

For example: nn.Linear(100,50) You have 100 input features and 50 outputs. Would this be considered as the input layer or already as a kind of second layer? The problem: I don't want to apply dropout ...
0
votes
0answers
89 views

Whats the difference between add.LSTM(num_hidden, droput=0.5) and add.Dropout(0.5) in Keras?

Could anyone please explain what is the difference between these two cases, specified in the title. I believe I am not the only one who is confused. I have read that it is preferrable to add Dropout ...
0
votes
1answer
29 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...