In my nonlinear dynamics class in college, we discussed a simple perceptron with two input neurons and one output neuron that is trained on the patterns
pattern | Input | Output 1 | 1,0 | 2 2 | 3,1 | 6 3 | 0,0 | 0
Solving the system of equations to determine the weights
$$w_1(1) + w_2(0) = 2$$ $$w_1(3) + w_2(1) = 6$$ $$w_1(0) + w_2(0) = 0$$
gives $w_1 = 2$ and $w_2 = 0$, which means the perceptron has made an abstraction. In more advanced neural networks, (a) how do you test for abstractions in your network and (b) how do you interpret what that abstraction means?