New answers tagged backpropagation
0
The ReLU (or "clipping negative values") is defined as $f(x) = max(0, x)$ so it's derivative is
$$
f'(x) =
\begin{cases}
0 &\quad\text{if}\qquad x < 0 \\
\text{undefined} & \quad \text{if} \qquad x = 0 \\
1 &\quad\text{if }\qquad x > 0\\
\end{cases}
$$
To deal with the undefined case you usually ...
0
Maybe it is easier to understand the derivative of pooling layer after we write down the matrix format of function max{x}=xW, where x is a tensor. Let us see an example with four different values, x = [x1, x2, x3, x4] and x1 is the largest value, such that max{x} = x1. Now, max{x} can be written down as matrix multiplication, such that xW or Wx', where W = [...
Top 50 recent answers are included
