# Neural network does not converge with negative symbols

I've created a simple 2-2-1 feedforward ANN to predict an XOR using Keras.

The activation function I'm using on all layers is a tanh, so in order to make use of the entire range of the function, i.e. [-1, 1], I've decided to use -1 instead of 0 as the symbol.

My input data is, thus, [[-1, -1], [-1, 1], [1, -1], [1, 1]], for an output of [[-1], [1], [1], [-1]].

I thought this would give better results since I'm using the entire range of the function and it supposedly would converge better because of that. Also, since I'm just using a different symbol, it should be the same as with using 0 and 1.

However, my network can not converge (giving a 0.5 accuracy), and what baffles me the most is that using 0 and 1 as symbols converges, and at a much faster rate.

Is there a reason for such a counter-intuitive (at least in my conception) thing to be happening?

I tried this experiment and was able to get some positive results. I will describe what I tried then perhaps you can specify where the differences may lie and we can further explore them. From what I tried I would assume that you are simply not training long enough.

# Creating the data

import numpy as np

n = 100000
x_train = np.zeros((n,2))
y_train = np.zeros((n,))
for i in range(n):
x_train[i,0] = np.random.choice([-1,1])
x_train[i,1] = np.random.choice([-1,1])
if x_train[i,0] == 1 and x_train[i,1] == 1 or x_train[i,0] == -1 and x_train[i,1] == -1:
y_train[i] = -1
else:
y_train[i] = 1

x_train = x_train.reshape(n, 2,)

n = 1000
x_test = np.zeros((n,2))
y_test = np.zeros((n,))
for i in range(n):
x_test[i,0] = np.random.choice([-1,1])
x_test[i,1] = np.random.choice([-1,1])
if x_test[i,0] == 1 and x_test[i,1] == 1 or x_test[i,0] == -1 and x_test[i,1] == -1:
y_test[i] = -1
else:
y_test[i] = 1

x_test = x_test.reshape(n, 2,)

print(x_test[0].T)
print(y_test[0])


[ 1. 1.]
-1.0

# Build the model

As you describe the model is 2 input nodes, 2 hidden nodes and 1 output node. Every node is using tanh as its activation function.

input_shape = (2,)

model = Sequential()
input_shape=input_shape))

model.compile(loss=keras.losses.mean_squared_error,
metrics=['accuracy'])


# Train the model

Because I generated many instances of the data I am only training for 10 epochs. However, if your input space is only the four possible inputs you may want thousands of epochs. Neural networks do take a long time to converge.

epochs = 10
batch_size = 128
# Fit the model weights.
history = model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))


Epoch 10/10
100000/100000 [==============================] - 1s 9us/step - loss: 9.3983e-05 - acc: 1.0000 - val_loss: 7.9096e-05 - val_acc: 1.0000