# How to do Back Propagation Updation for below code?

Below is the code by Siraj Raval for implementing Neural Networks from Scratch. I have some doubts regarding the Code:

1. Why during updation he did W2 = W2 + L1.T.dot(L2_Delta). I Mean shouldn't it be W2 = W2 - L1.T.dot(L2_Delta) as we should move in the opposite direction of the gradient as in Gradient Descent

2. According to the fundamental equation of Backpropagation The Update should be done as Shown in equation BP2 but in code, but the order is opposite in the code ??

L1_Delta =(L2_Delta.dot(W2.T)) * Sigmoid(L1,Deriv = True) # Hidden Layer Error

Similarly for Equation BP4 the Gradient

W2 += L1.T.dot(L2_Delta)

I think It Should be :

W2 += (L2_Delta).dot(L1.T)

# A Three Layer Model Layer 1: Input Layer
# Layer 2 = Hidden layer , Layer 3 = Output layer
#nO.OF wEIGHTS = #lAYERS - 1
import numpy as np

def Sigmoid(Z,Deriv = False):
P = np.reciprocal(1+np.exp(-Z))
if Deriv == False:
return P
else:
return Z*(1-Z)

#Input Data
X = np.array([[0,0,1],[0,1,1],[1,0,1],[1,1,1]])
Y = np.array([,
,
,
])

np.random.seed(1)
W1 = 2*np.random.random((3,4)) - 1
W2 = 2*np.random.random((4,1)) - 1

#Training

for j in range(60000):
L0 = X  #Input Layer
L1 = Sigmoid(np.dot(L0,W1))  #First Hidden Layer
L2 = Sigmoid(np.dot(L1,W2))  # Output Layer

L2_Error = Y - L2

if j % 1000 == 0:
print("Error: " + str(np.mean(np.abs(L2_Error))))

L2_Delta = L2_Error * Sigmoid(L2,Deriv = True)   #Output Layer Error
#Now How much layer 1 contributed in the error of Layer 2
L1_Error = L2_Delta.dot(W2.T)
L1_Delta = L1_Error * Sigmoid(L1,Deriv = True)  # Hidden Layer Erro