0
$\begingroup$

In a neural network, does gradient vanish during a great number epochs as well, rather that only vanishing through different layers?

$\endgroup$
2
  • $\begingroup$ What do you mean by "rather that only vanishing by different layers"? $\endgroup$ – Ethan Yun Mar 11 at 19:49
  • $\begingroup$ Sorry, I mean "Vanishing through different layers" I expressed myself badly $\endgroup$ – Domenico Bagnato Mar 11 at 19:50
0
$\begingroup$

Gradient vanishing means drastic decrease of gradients when backpropagating through many layers. This problem is also known for recurrent neural networks as they are mathematically equivalent to very deep networks.

That is true that gradients decrease during training. However, that is not gradient vanishing. That is the sign that the network has been trained, i.e. that is what you normally expect in the end of network training.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.