I am having a problem understanding the cost function in a neural network. I have read many books and blog posts, but all of them describe that point in neural networks is to minimize the cost function (like sum squared error):
I tried to look at code for solving a problem with a multi layer neural network and back propagation. My question is: where in the code can I find the cost function? How can I plot the error surface?
import numpy as np X_XOR = np.array([[0,0,1], [0,1,1], [1,0,1],[1,1,1]]) y_truth = np.array([,,,]) def sigmoid(x): return 1 / (1 + np.exp(-x)) def sigmoid_der(output): return output * (1 - output) np.random.seed(1) syn_0 = 2*np.random.random((3,4)) - 1 syn_1 = 2*np.random.random((4,1)) - 1 for i in range(60000): layer_1 = sigmoid(X_XOR.dot(syn_0)) layer_2 = sigmoid(layer_1.dot(syn_1)) error = 0.5 * ((layer_2 - y_truth) ** 2) layer_2_delta = error * sigmoid_der(layer_2) layer_1_error = layer_2_delta.dot(syn_1.T) layer_1_delta = layer_1_error * sigmoid_der(layer_1) syn_1 -= layer_1.T.dot(layer_2_delta) syn_0 -= X_XOR.T.dot(layer_1_delta) if i % 10000 == 1: print(layer_2) print(layer_2)