# XOR problem with neural network, cost function

I am having a problem understanding the cost function in a neural network. I have read many books and blog posts, but all of them describe that point in neural networks is to minimize the cost function (like sum squared error):

I tried to look at code for solving a problem with a multi layer neural network and back propagation. My question is: where in the code can I find the cost function? How can I plot the error surface?

import numpy as np
X_XOR = np.array([[0,0,1], [0,1,1], [1,0,1],[1,1,1]])
y_truth = np.array([[0],[1],[1],[0]])

def sigmoid(x):
return 1 / (1 + np.exp(-x))
def sigmoid_der(output):
return output * (1 - output)

np.random.seed(1)
syn_0 = 2*np.random.random((3,4)) - 1
syn_1 = 2*np.random.random((4,1)) - 1

for i in range(60000):
layer_1 = sigmoid(X_XOR.dot(syn_0))
layer_2 = sigmoid(layer_1.dot(syn_1))
error = 0.5 * ((layer_2 - y_truth) ** 2)
layer_2_delta = error * sigmoid_der(layer_2)
layer_1_error = layer_2_delta.dot(syn_1.T)
layer_1_delta = layer_1_error * sigmoid_der(layer_1)
syn_1 -= layer_1.T.dot(layer_2_delta)
syn_0 -= X_XOR.T.dot(layer_1_delta)
if i % 10000 == 1:
print(layer_2)

print(layer_2)
• @JahKnows can you explain? – lukassz May 16 '18 at 17:07

The cost function can be found in the delta rule, meaning the way you calculate your deltas. This delta is nothing more than the derivative of your error function after the weights: $\frac{\partial E}{\partial w_{ij}}$. So, if you are just interested in where the cost is encoded, this is the answer you are looking for.