New answers tagged

0

The ReLU (or "clipping negative values") is defined as $f(x) = max(0, x)$ so it's derivative is $$ f'(x) = \begin{cases} 0 &\quad\text{if}\qquad x < 0 \\ \text{undefined} & \quad \text{if} \qquad x = 0 \\ 1 &\quad\text{if }\qquad x > 0\\ \end{cases} $$ To deal with the undefined case you usually ...


0

Maybe it is easier to understand the derivative of pooling layer after we write down the matrix format of function max{x}=xW, where x is a tensor. Let us see an example with four different values, x = [x1, x2, x3, x4] and x1 is the largest value, such that max{x} = x1. Now, max{x} can be written down as matrix multiplication, such that xW or Wx', where W = [...


Top 50 recent answers are included