1
Neural Networks try to approximate the function which maps the given image to its label. Changing the number of hidden layers essentially means changing the number of parameters, which would be used to approximate that input-label mapping. These parameters include weights and biases ( in the case of Dense layers ).
Depending upon the number of parameters, ...
1
I now have a working network. It turned out that the gradients were all zeros after only about 3000 update step. I tried two approaches to fix this - using Batch Normalization after each activation function in the feed-forward net and changing the activation function from ReLU to Leaky ReLU. Both worked, and I ended up using the Leaky ReLU without ...
1
I've been also looking for the answer of this question, and I give my different view of Gumbel softmax just because I think this is a good question.
From a general point of view:
We use softmax normally because we need a so-called score, or a distribution $\pi_1 .. \pi_n$ for representing n probabilities of categorical variable with size n;
We use Gumbel-...
Only top voted, non community-wiki answers of a minimum length are eligible
