I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python.
I have four training data points (X). As they are used for supervised learning so, each data point has its corresponding correct output pair (D). I have implemented SGD and used generalized Delta rule (wij ← wij + α δixj). I have trained my perceptron 10,000 times (epochs= 10,000).
Although everything looks fine to me, I don't get the right results when I test it with test values. I need some suggestions so that I can improve my results on test data. P.S. How can I improve this code?
Code
import numpy as np
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def Delta_SGD(W, X, D):
N = 4
for x in range(N):
v1 = np.dot(X[x][0], W[0])
v2 = np.dot(X[x][1], W[1])
v3 = np.dot(X[x][2], W[2])
#weighted sum
V = v1+v2+v3
#output of neuron
y = sigmoid(V)
#error
e = D[x] - y
#derivative of sigmoid(y)
delta = (y)*(1-y)*e
#Delta rule
DW = alpha*delta*X[x]
#updated weights
W[0] = W[0] + DW[0]
W[1] = W[1] + DW[1]
W[2] = W[2] + DW[2]
return W
#input data points
X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1] ])
#Correct output pairs
D = np.array([[0,0,1,1]]).T
#learning rate
alpha = 0.9
#random weights
W = 2*np.random.random((3,1)) - 1
#10000 epochs
for epoch in range(10000):
W = Delta_SGD(W, X, D)
print(epoch)
#Final weights after all epochs
print("Final weights are \n", W)
#testing network
N = 4
for x in range(N):
v1 = np.dot(X[x][0], W[0])
v2 = np.dot(X[x][1], W[1])
v3 = np.dot(X[x][2], W[2])
V = v1+v2+v3
y = sigmoid(V)
print("output of neuron is \n ", y)