# Tag Info

The ReLU (or "clipping negative values") is defined as $f(x) = max(0, x)$ so it's derivative is $$f'(x) = \begin{cases} 0 &\quad\text{if}\qquad x < 0 \\ \text{undefined} & \quad \text{if} \qquad x = 0 \\ 1 &\quad\text{if }\qquad x > 0\\ \end{cases}$$ To deal with the undefined case you usually ...