How does neural network solve XOR problem

Question

I'm reading a wonderful tutorial about neural network. This is the best tutorial I've ever seen but I can't understand one thing as below:

In the link above, it is talking about how the neural work solves the XOR problem.

It says that we need two lines to separate the four points. But I don't know the second table.

XOR:

input1    input2    output
  0         0         0
  0         1         1
  1         0         1
  1         1         0

First table:

input1    input2    output
  0         0         0
  0         1         1
  1         0         1
  1         1         1

In my opinion, the first table is OK because it includes the XOR, which means that what the second table need to do is to remove the forth input. So I think the second table should be as below:

input1    input2    output
  0         0         1
  0         1         1
  1         0         1
  1         1         0

How in the link it says the second table is like this:

input1    input2    output
  0         0         0
  0         1         0
  1         0         0
  1         1         1

In a word, I can understand why the single layer neural network can't solve the XOR problem but I can't understand how the two layers neural network work to solve it.

Answer 1

Notice that the first table (orange line) is performing an OR operation and the second table (blue line) is performing an AND operation. XOR can be defined as (x OR y) AND NOT (x AND y) or $(x \lor y) \land \lnot (x \land y)$, so in other words: orange should fire and blue shouldn't fire. If we now look at this figure from your tutorial:

we can see that the weight of blue in the second layer is negative and small enough (more negative) such that the output can never fire if blue fires, i.e. the output can't fire if both inputs are firing.

$0 + 0 \ngtr 1 : \emptyset $ shouldn't fire

$-2 + 0 \ngtr 1 : (x \land y) = T$ shouldn't fire

$0 + 1.1 \gt 1 : (x \lor y) \land \lnot (x \land y) = T$ should fire

$-2 + 1.1 \ngtr 1 : (x \land y) \land (x \lor y) = T$ shouldn't fire

Answer 2

The table that you are referring to is doing OR operation. whenever you have just a neuron in your net you are able to have one line to separate your data. but for xor data you have to have two line separators. it is common for solving this problem to have two neurons in the first layers which both do OR operation and in the second layer to have one neuron to do and operation. if you stack these two layers, you will get the result. I suggest you to see the shape of this to see the nonlinear decision boundary. Also the second table is doing AND operation.