I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this topic.
I know a perceptron with no hidden layers can be seen as defining a hyperplane in between two classes, which works fine if they are linearly separable.
However, I find more difficult to understand what exactly does a multi layer perceptron do, and how it can classify an input that is initially not linearly separable.
I found this page http://www.borgelt.net/doc/mlpd/mlpd.html that says the hidden layers can be interpreted as making a coordinate transformation on the input. I would like to read more about this, but I didn't find much online, so I would really appreciate if you could elaborate on this or point me to some paper or book that covers this.
I am also puzzled by the visualisation that appears here http://tiny.cc/2l7o7y , a 3 neuron hidden layer manages to separate the input, which makes perfect sense since separating that input in a 3 dimensional space is trivial. However, adding another 2 neuron layer after that creates in them shapes that are obviously non-linear. My guess is that it represents the separation created in the original 2-dimensional space. Is that right? If not, what are we seeing here, exactly?