What is the reasoning behind ReLu boundary lines appearing linear when plotting in 2D? Does this generalize to higher dimensions in that boundary lines in large dimensions are linear hyperplanes as well?
For instance, in tensorflow's playground any time ReLu is chosen as the activation function, the resulting boundary line is made of lines.
I'm failing to see the connection between the underlying neurons outputting only z or 0 and how that connects to the output plot.
Thanks very much for any thoughts/help!