I want to create a simple perceptron with three inputs. $ \sigma(w_1x_1 + w_2x_2 + w_3x_3) $, where $ \sigma = \frac{1}{1 + e^{-x}.} $ $$ L(w) = \sum_i^3(y_i -\sigma(w_1x_1 + w_2x_2 + w_3x_3)); $$
$ \large \frac{dL}{dw} =2(y - \sigma(w_1x_1 + w_2x_2 + w_3x_3))\sigma(1 - \sigma)x. $
I am trying to program reverse propagate the error. Help, what am I doing wrong.
def sigmoid(x):
return 1/(1 + np.exp(-x))
data_train = np.array([[1, 0, 1], [0, 1, 1], [0, 1, 0], [0, 0, 1]])
train_res = np.array([[1], [0], [0], [0]])
weights = 2*np.random.random((3,1)) - 1
for i in range(10000):
out = sigmoid(np.dot(data_train,weights))
dE = 2*np.dot(data_train,np.dot((out - train_res),np.dot(out,(1 - out))))
weights += dE
test = np.array([0, 0, 1])
print(sigmoid(np.dot(test, weights)))