0
$\begingroup$

I am implementing a Multiclass neural network that will recognize digits from the input images. The input images are five by five pixels which display five numbers. So, the neural network model contains 25 input nodes. 4 output nodes (as we have to classify five outputs) and 50 hidden nodes. I am using sigmoid function as an activation function for hidden nodes and softmax function as activation function for the output nodes.
My input argument X contains stacked two-dimensional image data i.e. 5*5*5 three dimensional matrices. So, I have converted into a (25,) vector using np.reshape(X[:,:,x],25) so that I can get the weighted sum easily (as the weights are 25*50 for input to hidden nodes and 50*5 for hidden to output nodes).

The problem I am facing is that when I calculate the Delta for input to hidden and hidden to output respectively, I have value error "ValueError: shapes (50,) and (5,5) not aligned: 50 (dim 0) != 5 (dim 0)" which I totally understand is because of the dimensions of both the arrays. But I am unable to figure out the solution so, that I can reshape them some way and take dot products. As I am learning python and this neural network stuff for the first time and writing my own code so, I don't have much expertise in playing with matrices. I need some help in 1) fixing this problem, 2) how can I improve this network (Future practices). 3) How can I make the network generic so that if I can add more layers etc I don't mess up with matrix multiplication with different dimensions? A prototype of code is below.

Code

# other stuff
def function(W1, W2, X, D):
 N = 5
 for x in range(N):
    # reshaping input in 25*1 vector
    l0 = np.reshape(X[:,:,x],25)
    ll = sigmoid(np.dot(l0,W1))
    l2 = softmax(np.dot(ll,W2))
    l2_error = D - l2
    l2_delta = l2_error
    l1_error = l2_delta.dot(W2.T)
    l1_delta = ll*(1-ll)*l1_error
    DW2 = alpha*ll.T.dot(l2_delta) #ValueError: shapes (50,) and (5,5) not aligned: 50 (dim 0) != 5 (dim 0)
    W2 = W2 + DW2
    DW1 = alpha*l0.T.dot(l1_delta) #ValueError: shapes (25,) and (5,50) not aligned: 25 (dim 0) != 5 (dim 0)
    W1 = W1 + DW1

# other stuff

X = np.zeros((5,5,5), dtype=int)
D = np.zeros((5,5), dtype=int)

X[:,:,0] = [[0 ,1, 1, 0, 0],
       [0 ,0, 1, 0, 0],
       [0 ,0, 1, 0, 0],
       [0 ,0, 1, 0, 0],
       [0 ,1, 1, 1, 0]]

X[:,:,1] = [[1 ,1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [0 ,1, 1, 1, 0],
       [1 ,0, 0, 0, 0],
       [1 ,1, 1, 1, 1]]

X[:,:,2] = [[1 ,1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [0 ,1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [1 ,1, 1, 1, 0]]

X[:,:,3] = [[0 ,0, 0, 1, 0],
       [0 ,0, 1, 1, 0],
       [0 ,1, 0, 1, 0],
       [1 ,1, 1, 1, 1],
       [0 ,0, 0, 1, 0]]

X[:,:,4] = [[1 ,1, 1, 1, 1],
       [1 ,0, 0, 0, 0],
       [1, 1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [1 ,1, 1, 1, 0]]

D = np.array([ [1,0,0,0,0],[0,1,0,0,0],[0,0,1,0,0],[0,0,0,1,0],[0,0,0,0,1] ]).T

W1 = 2*np.random.random((25,50)) - 1
W2 = 2*np.random.random((50,5)) - 1
W1, W2 = Multiclass(W1, W2, X, D)
$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.