New answers tagged backpropagation
0
According to the Cambridge Dictionary saturation means
the act or result of filling a thing or place completely so that no
more can be added
In this context, it refers to a function for which a bigger input will not lead to a relevant increase in output. So if the gradient is saturated (meaning it is extremely close to zero), a bigger upstream gradient ...
Top 50 recent answers are included
