New answers tagged backpropagation
0
According to the Cambridge Dictionary saturation means
the act or result of filling a thing or place completely so that no
more can be added
In this context, it refers to a function for which a bigger input will not lead to a relevant increase in output. So if the gradient is saturated (meaning it is extremely close to zero), a bigger upstream gradient ...
Top 50 recent answers are included
Related Tags
backpropagation × 249neural-network × 145
machine-learning × 81
deep-learning × 54
gradient-descent × 43
cnn × 23
lstm × 17
convolutional-neural-network × 17
python × 13
rnn × 12
convolution × 12
loss-function × 11
training × 9
tensorflow × 8
optimization × 8
activation-function × 7
reinforcement-learning × 6
recurrent-neural-net × 6
math × 6
cost-function × 6
keras × 5
pytorch × 5
numpy × 5
batch-normalization × 4
softmax × 4