# Matrix multiplication issue (shapes not alligned)

I am building an RNN using numpy only and have started on the forward propagation section. However i am having some issues aligning my matrices. The issue is on this line:

h = np.dot(u, x) + np.dot(aprev, w) + bh


More specifically, the problem is with this part:

np.dot(u, x)


I tried playing around with it by transposing different parts but I still receive the error:

ValueError: shapes (8,8) and (4,8) not aligned: 8 (dim 1) != 4 (dim 0)


How can i overcome this? The entirety of my code is pasted below, and note that at the beginning i transposed x, to make data entry and inserting data into its corresponding timestep easy. '.T' can be easy to miss sometimes.

import numpy as np

# Data Processing

x = np.array([
# t/no. of inputs
[1, 2, 3, 4, 5, 6],
[7, 8, 9, 10, 11, 12],     # Samples
[13, 14, 15, 16, 17, 18],
[19, 20, 21, 22, 23, 24]]).T

# Model Parameters

numInputs = x.shape
timeSteps = x.shape
numNeurons = 8

u = np.random.random((numNeurons, numInputs))
v = np.random.random((numInputs, numNeurons))
w = np.random.random((numNeurons, numNeurons))
bh = np.zeros((numNeurons, 1))
bo = np.zeros((numInputs, 1))
aprev = 0

# Training

def fprop(x, aprev, u, v, w, bh, bo):
h = np.dot(u, x) + np.dot(aprev, w) + bh
a = np.tanh(h)
o = np.dot(a, v) + bo
yhat = (np.exp(o))/(np.sum(np.exp(o)))
aprev = a

for t in x:
fprop(t, aprev, u, v, w, bh, bo)

• Are you familiar to matrix multiplication? – Media Aug 19 '18 at 18:47
• @Media I have been using dot product because I’ve seen it being used in many other examples. Does matrix multiplication do the same thing and overcome my problem as well? – Finn Williams Aug 19 '18 at 18:50
• It depends on your task, do you want to have matrix multiplication? For matrix multiplication your dimensions must match. – Media Aug 19 '18 at 18:57
• @Media it’s a toy data set. I’m learning how to program an RNN from scratch, so I need to stay away from automatic libraries like tensorflow – Finn Williams Aug 19 '18 at 19:15
• You changes the order from u,x to x,u which are completely different operations. np.dot(a,b) should be the same as a.dot(b), but not b.dot(a) – keiv.fly Aug 19 '18 at 23:00